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EXECUTIVE  SUMMARY 


During  simultaneous  parallel  approaches.  Federal  Aviation  Administration  (FAA) 
regulations  require  air  traffic  controllers  (ATC)  to  staff  special  radar-monitoring  positions 
dedicated  to  maintaining  separation  between  adjacent  aircraft.  While  the  monitor  controller 
response  to  a  deviation,  or  blunder,  towards  the  adjacent  approach  is  critical,  the  response  of  the 
threatened  aircrew  and  aircraft  also  affects  the  safety  of  the  operation. 

Two  studies  were  conducted  during  the  Precision  Runway  Monitor  (PRM) 
Demonstration  Program  to  measure  Boeing  727  (B727)  and  McDonnell  Douglas  10  (DC  10) 
performance  during  ATC-directed  breakouts  given  then-existing  pilot  training  and  ATC  breakout 
phraseology.  A  follow-on  study  measured  B727  breakout  performance  given  increased  pilot 
situational  awareness.  The  recorded  B727  and  DCIO  simulator  tracks  were  then  incorporated 
into  the  PRM  Program’s  risk  assessment  model  as  representative  threatened-aircraft  breakouts. 
All  the  B727  and  DCIO  tracks  recorded  during  the  breakout  response  studies  were  used  in  one 
analysis.  Based  on  the  results  of  the  follow-on  study,  B727  and  DCIO  tracks  with  very  slow  turn 
performance  were  excluded  from  a  second  analysis.  The  two  risk  analysis  results  suggested  that 
then-existing  procedures  were  not  sufficient  to  meet  the  target  level  of  safety  and  that  a  pilot 
familiarization  program  was  needed  for  close  parallel  approaches. 

The  PRM  Demonstration  Fhogram  included  other  simulations  that  incorporated  remote- 
site  McDonnell  Douglas  80  (MD80)  cockpit  simulators.  Simultaneous  parallel  approach  studies 
were  also  conducted  by  the  Multiple  Parallel  Approach  Procedure  (MPAP)  technical  working 
group.  The  latter  studies  incorporated  newer  aircraft  types  such  as  the  Boeing  747-400 
(B747-400).  Observations  made  during  both  programs  indicated  that  some  aircrews  had 
difficulty  executing  the  breakout  when  the  autopilot  was  connected,  and  that  some  newer  aircraft 
types  had  longer  breakout  response  times  than  the  older,  analog  aircraft. 

FAA  commissioned  two  studies  to  evaluate  the  breakout  performance  of  two  advanced- 
avionics  aircraft:  the  B747-400  and  Airbus  320  (A320).  This  report  documents  the  A320  study 
conducted  in  1995  using  a  full-motion  A320  cockpit  simulator  at  Northwest  Aerospace  Training 
Corporation  in  Eagan,  Minnesota.  Subjects  were  qualified  commercial  A320  pilots.  The 
primary  purpose  of  the  Study  was  to  evaluate  A320  breakout  response  given  current  procedures 
and  given  proposed  training  materials.  The  secondary  purpose  was  to  elicit  pilot  preferences 
regarding  breakout  procedures  and  the  tested  training  material. 

TEST  DESIGN 

Phase  1  of  the  study  evaluated  A320  breakout  performance  given  current  procedures  and 
pilot  training.  The  ATC  phraseology  was 

(Aircraft)  TURN  (left/right)  IMMEDIATELY  HEADING  (degrees). 

(Climb/Descend)  AND  MAINTAIN  (altitude). 

Phase  2  tested  the  combined  effect  of  increased  situational  awareness,  a  written  breakout 
procedure,  and  proposed  ATC  breakout  phraseology  on  A320  breakout  performance.  The 
situational  awareness  training  package  included  a  video  that  explained  close  parallel  approach 
operations  and  what  is  expected  of  the  aircrew,  an  1 1-0  Airport  Information  Page,  and  a  pilot 
awareness  training  bulletin.  The  breakout  procedure  was  A320-specific  and  encouraged  "open 
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climb"  rather  than  take  off/go-around  (TOGA)  thrust  for  climbing  breakouts  above  400  feet 
above  ground  level  (AGL).  The  pilots  did  not  practice  the  procedure  beforehand.  The  ATC 
phraseology  was 

(Aircraft)  TRAFFIC  ALERT.  (Aircraft)  TURN  (left/right)  IMMEDIATELY 

HEADING  (degrees).  (Climb/Descend)  AND  MAINTAIN  (altitude). 

ATC-directed  climbing  breakouts  during  manual  and  autopilot-coupled  approaches  were 
tested  at  decision  altitude,  500  feet  above  ground  level  (AGL),  and  1800  feet  AGL.  Descending 
breakouts  at  1800  feet  AGL  were  also  tested  for  both  approach  modes.  Phase  1  included  an 
engine-out  distraction  at  decision  altitude.  Phase  2  included  climbing  breakouts  at  1000  feet 
AGL.  The  test  criteria  for  both  phases  were  based  on  the  DC  10  and  B727  data  used  in  the  PRM 
risk  analysis;  mean  time  to  start  of  turn  less  than  or  equal  to  8  seconds  and  maximum  time  to 
start  of  turn  of  less  than  17  seconds. 

TEST  RESULTS 

Breakout  performance  during  Phase  1  did  not  meet  the  test  criteria  for  either  approach 
mode.  Except  for  two  combinations  of  approach  mode  and  breakout  location,  mean  times  to 
start  of  turn  were  greater  than  8  seconds.  In  addition,  time  to  start  of  turn  was  greater  than  17 
seconds  in  16  out  of  136  trials.  The  pilots  agreed  that  breakout  procedure  training  and  better 
situational  awareness  would  have  enhanced  their  performance. 

Mean  time  to  start  of  turn  was  acceptable  for  both  approach  modes  in  Phase  2,  although 
mean  time  to  start  of  turn  following  manual  approaches  was  1  to  3  seconds  less  than  mean  time 
following  autopilot-coupled  approaches.  Time  to  start  of  turn  was  greater  than  17  seconds  in 
three  trials.  Two  factors  were  associated  with  these  three  trials:  breakouts  at  decision  altitude 
and  individual  pilot  response  time.  Eighty-eight  to  92  percent  of  the  pilots  agreed  that  the 
training  aids  increased  their  awareness  of  simultaneous  close  parallel  approaches.  Approval  for 
the  tested  ATC  phraseology  was  similar,  with  75  percent  strongly  agreeing  that  the  ATC 
phraseology  encouraged  a  quicker  response  than  the  current  phraseology.  Based  on  their 
experience,  75  percent  of  the  pilots  preferred  using  the  autopilot  during  simultaneous  close 
parallel  approaches  in  Category  I  weather  but  only  56  percent  preferred  using  the  autopilot 
during  the  breakout.  This  result  did  not  agree  with  their  actual  performance  because  the 
autopilot  remained  engaged  during  most  of  the  Phase  2  autopilot-coupled  trials.  The  autopilot 
was  disconnected  before  the  turn  in  only  seven  percent  of  the  trials,  and  all  except  one 
disconnect  occurred  with  one  crew.  Several  pilots  commented  that  they  did  not  like  using  "open 
climb"  at  any  altitude  and  felt  that  TOGA  thrust  was  better.  Several  pilots  stated  they  did  not 
like  the  procedure  to  call  out  when  the  aircraft  was  400  feet  AGL  because  it  interfered  with  the 
callout  at  200  feet  above  minimums. 

In  the  Phase  1  surveys,  six  subject  crews  mentioned  TOGA  thrust  as  an  aircraft  limitation 
that  could  cause  an  inherent  unwanted  delay.  They  wrote  that  if  heading  is  manually  changed 
before  the  thrust  levers  are  moved  to  TOGA  then  the  avionics  revert  to  the  go-around  track  and 
the  pilots  have  to  enter  the  heading  again.  This  phenomenon  was  observed  in  14  of  the  autopilot- 
coupled  tracks.  The  TOGA-induced  roll  back  towards  the  go-around  track  added  4  to  18  seconds 
to  these  turn  responses. 


In  the  Phase  2  surveys,  several  pilots  commented  that  the  autothrottle  was  slow  to 
respond  during  open  climb.  Their  observations  were  consistent  with  the  data  analysis  results. 
Although  using  open  climb  did  not  appear  to  affect  turn  performance,  it  significantly  increased 
the  mean  time  required  for  the  aircraft  to  transition  from  descent  to  climb.  In  addition,  the 
transition  time  was  longer  than  12  seconds  for  many  open-climb  breakouts  at  1800  feet  AGL. 
Transition  times  appeared  to  be  related  to  engine  rpm  at  the  start  of  the  response. 

CONCLUSIONS 

Three  conclusions  were  made  regarding  A320  breakout  performance.  First,  turn 
performance  did  not  meet  the  test  criteria  given  current  training  and  ATC  phraseology.  Second, 
turn  performance  did  meet  the  test  criteria  when  the  pilots  received  awareness  and  procedure 
training  combined  with  proposed  ATC  breakout  phraseology.  There  were  a  few  long  response 
times,  but  limiting  breakouts  below  400  feet  AGL  could  significantly  reduce  the  frequency  of 
their  occurrence.  Third,  the  tested  breakout  procedure  should  be  refined  because  it  conflicted 
with  other  cockpit  procedures  and  increased  the  time  to  transition  to  a  positive  climb  rate. 

Based  on  these  conclusions,  it  is  recommended  that  a  combination  of  pilot  situational 
awareness  training,  A320  breakout  procedure  and  modified  ATC  breakout  phraseology 
equivalent  to  that  used  in  Phase  2  be  employed  for  simultaneous  parallel  approach  operations  in 
instrument  meteorological  conditions. 
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1.  INTRODUCTION 


Since  1987,  the  Federal  Aviation  Administration  (FAA)  has  initiated  two  programs  with 
the  purpose  of  increasing  airport  capacity  during  instrument  meteorological  conditions  (IMC)  to 
levels  near  visual  meteorological  condition  (VMC)  capacity  while  maintaining  the  safety  of  the 
operations.  One  program  was  the  Precision  Runway  Monitor  (PRM)  Demonstration  Program, 
which  concluded  in  1991  with  the  recommendation  to  allow  simultaneous  instrument  landing 
system  (ILS)  approaches  to  dual  parallel  runways  spaced  3400  to  4300  feet  apart,  given  certain 
equipment  and  operational  requirements  [1].  The  other  is  the  Multiple  Parallel  Approach 
Program  (MPAP),  an  on-going  FAA  effort  that  started  in  1988.  The  MPAP  technical  working 
group  (TWG)  has  developed  and  tested  new  procedures  for  simultaneous  dual,  triple,  and 
quadruple  parallel  approaches  in  IMC  utilizing  existing  and/or  new  technology  along  with 
proposed  air  traffic  control  (ATC)  procedures  [2,  3]. 

During  simultaneous  ILS  approaches,  FAA  regulations  require  air  traffic  controllers  to 
staff  a  special  radar-monitoring  position  dedicated  to  maintaining  separation  between  aircraft.  If 
an  aircraft  wanders  or  blunders  into  the  no  transgression  zone  (NTZ),  then  the  monitor  controller 
is  required  to  break  any  threatened  aircraft  out  of  its  approach  path  and  away  from  the  blundering 
aircraft.  The  ATC-directed  breakout  scenario  is  described  in  more  detail  in  Section  1.1.1. 

While  the  monitor  controller  response  to  a  blunder  is  critical  to  the  safety  of  simultaneous 
parallel  ILS  approaches,  the  responses  of  the  threatened  aircrew  and  aircraft  also  affect  the  safety 
of  the  operation.  If  the  combined  pilot/aircraft  response  to  the  ATC-directed  breakout  is  not 
sufficient  in  either  time  or  magnitude  then  a  near  miss  or  mid-air  collision  could  occur.  Thus,  it 
is  important  to  know  the  limitations  of  the  threatened  crew/aircraft  response  for  the  major  aircraft 
types. 

There  is  a  concern  with  newer  advanced  avionics  aircraft  that  the  flight  automation  is  not 
designed  for  emergency  maneuvers  in  the  terminal  area.  In  order  to  evaluate  breakout 
performance  in  newer  aircraft,  FAA  contracted  for  two  breakout  performance  studies  using 
digital  aircraft  types  with  advanced  avionics  systems:  the  Airbus  320  (A320)  and 
Boeing  747-400  (B747-400).  This  report  documents  the  results  of  the  A320  study.  A  separate 
report  [4]  documents  the  results  of  the  B747-400  study. 

The  remainder  of  this  section  provides  details  about  previous  performance  studies,  the 
PRM  Demonstration  Program,  and  the  Multiple  Parallel  Approach  Program.  Section  2  details 
the  experimental  designs  used  in  the  A320  study.  Section  3  describes  the  test  and  analysis 
procedures  for  the  A320  study.  Section  4  presents  the  breakout  performance  analysis,  and 
Section  5  presents  pilot  survey  results.  Section  6  summarizes  the  findings. 

1.1  BACKGROUND 

1.1.1  Simultaneous  Parallel  Approach  Blunder  Scenario 

This  section  describes  the  events  that  occur  when  an  aircraft  on  a  simultaneous  parallel 
approach  deviates  towards  an  adjacent  approach.  A  blunder  occurs  when  the  deviation  is 
unexpected  and  sufficient  to  require  human  intervention.  A  deviation  in  which  the  aircraft 
continues  towards  the  adjacent  approach  without  responding  to  the  intervention  is  called  a  worst- 
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case  blunder.  An  ATC-directed  breakout  is  a  go-around  executed  by  an  aircraft  on  the  adjacent 
approach  because  of  the  blunder. 

During  simultaneous  parallel  approaches,  aircraft  along  the  final  approach  courses  are 
monitored  by  air  traffic  controllers  at  special  radar  positions.  There  is  one  monitor  controller  and 
display  for  each  parallel  approach  course.  For  dual  parallel  approaches  at  runway  separations  of 
4300  feet  or  greater,  the  ARTS  radar  displays  depict  a  2000-foot  wide  no  transgression  zone 
(NTZ)  centered  between  the  runways.  For  multiple  parallel  approaches  and  for  dual  parallel 
approaches  with  separations  between  3400  feet  and  4300  feet,  the  NTZ  is  depicted  on  a  special 
high-resolution  color  display.  In  addition,  the  color-display  software  includes  alert  logic  to 
detect  when  an  aircraft  is  deviating  towards  the  NTZ. 

Figure  1-1  depicts  the  sequence  of  events  during  a  worst-case  blunder  event  on  close 
parallel  approaches  (runway  separation  between  3400  and  4300  feet).  The  blunder  begins  when 
an  aircraft  starts  deviating  towards  the  adjacent  approach  (1).  A  caution  alert  is  generated  at  (2), 
and  the  monitor  controller  decides  to  break  the  adjacent  aircraft  out  of  the  approach  stream  at  (3). 
The  breakout  instruction  is  received  by  the  aircrew  at  (4),  and  the  breakout  maneuver  begins  at 
(5).  The  closest  point  of  approach  between  the  two  aircraft  occurs  at  (6),  after  which  separation 
increases. 


Figure  1-1.  Sequence  of  events  during  an  approach  blunder. 


2 


1.1.2  B727  and  DCIO  Performance  Studies 

FAA  Standards  Development  Branch  conducted  two  studies  in  1989  to  measure  aircrew 
performance  during  missed  approaches  and  ATC-directed  breakouts  [5,  6].  These  studies 
utilized  the  Boeing  727  (B727)  cockpit  simulator  at  the  Mike  Monroney  Aeronautical  Center, 
Oklahoma  City,  OK,  and  the  Federal  Express  McDonnell  Douglas  10  (DCIO)  cockpit  simulator 
at  Memphis,  TN.  Both  simulators  were  certified  equipment  with  six  degrees  of  motion.  Three 
groups  of  go-around  scenarios  were  evaluated:  missed  approaches  where  the  crew  turned  when 
able;  missed  approaches  where  the  crew  followed  company  policy;  and,  ATC-directed  breakouts 
at  low  and  high  altitudes.  The  results  of  all  three  scenario  groups  were  used  in  the  development 
of  the  obstacle  assessment  surfaces  required  in  FAA  Order  8260.41  [7].  The  ATC-directed 
breakout  scenarios  were  also  incorporated  into  the  PRM  Demonstration  Program  risk  assessment 
[1]. 

The  ATC-directed  breakouts  were  at  6  nautical  miles  from  the  threshold  (1800  feet  above 
ground  level  (AGL)),  at  200  feet  AGL  (Category  I  decision  altitude  (DA)),  and  at  100  feet  AGL 
(Category  n  DA).  In  addition,  two  trials  in  each  study  measured  breakout  performance  when  the 
crew  was  distracted.  Based  on  discussions  with  the  user  community,  engine  failure  was  selected 
as  a  representative  mechanical-failure  distraction.  The  ATC  phraseology  used  during  these  trials 
came  from  FAA  71 10.65F,  paragraph  5-126: 

(Aircraft)  TURN  (left/right)  HEADING  (degrees)  IMMEDIATELY, 

CLIMB  AND  MAINTAIN  (altitude). 

For  the  breakouts  at  decision  altitude,  the  pilots  generally  found  the  workload  to  be  more 
demanding  than  average  and  passenger  comfort  to  be  slightly  unacceptable.  The  DCIO  pilots 
also  felt  that  the  altitude  was  slightly  unsafe.  Some  DCIO  pilots  did  not  have  a  problem  with  the 
low-altitude  breakouts,  while  others  were  concerned  that  they  had  to  reconfigure  the  aircraft, 
thus  delaying  the  turn.  In  both  studies,  the  pilots  felt  that  knowledge  of  another  aircraft  on  a 
simultaneous  approach  in  instrument  meteorological  conditions  (IMC)  would  have  affected  their 
response.  Finally,  there  was  general  consensus  among  the  participating  pilots  and  controllers 
that  the  ATC  phraseology  needed  to  be  revised  to  more  clearly  reflect  the  urgency  of  the 
situation. 

For  the  ATC-directed  breakouts,  the  statistics  for  the  time  delay  from  start  of  the 
controller  breakout  instruction  to  the  start  of  the  turn  are  listed  in  Table  1-1.  Start  of  turn  was 
declared  when  the  aircraft  achieved  a  3-degree  roll  in  the  appropriate  direction.  In  general,  the 
pilots  were  able  to  initiate  the  turn  within  17  seconds;  a  few  pilots  exceeded  20  seconds.  The 
engine-out  distraction  added  2  to  3  seconds  to  the  average  response  time.  A  conclusion  of  the 
studies  was  that  the  long  response  times  might  have  been  due  to  various  interpretations  of  the 
meaning  of  the  word  "iimnediately." 

Because  of  the  long  response  times  observed  in  the  studies,  a  follow-on  study  was 
conducted  using  the  B727  simulator  [8].  In  this  study,  the  pilots  were  briefed  that  "immediately" 
meant  that  the  controller  had  observed  a  situation  that  required  an  urgent  response  from  the  pilot 
in  order  to  ensure  the  safety  of  the  aircraft.  In  addition,  the  ATC  phraseology  was  changed  to: 

(Aircraft)  TURN  (left/right)  IMMEDIATELY  HEADING  (degrees), 
CLIMB  AND  MAINTAIN  (altitude). 
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Table  1-1.  Time  from  Start  of  ATC  Instruction  to  Start  of  Turn  (seconds) 
from  the  B727  and  DC10  Studies* 


B727 

DCIO 

Scenario 

Mean 

Minimum 

Maximum 

Mean 

Minimum 

Maximum 

100  ft  AGL 

7.3 

2 

22 

6 

2 

15 

200ftAGL 

4.9 

2 

13 

3 

1 

17 

1800  ft  AGL 

4.5 

2 

16 

5 

1 

23 

Engine  Out 

7.7 

3 

18 

8 

1 

17 

Follow-on 

Study 

5.6 

2 

11 

- 

- 

*  Time  was  measured  in  1 -second  intervals. 


All  ATC-directed  breakouts  were  at  decision  altitude.  The  statistics  for  the  time  to  start 
of  turn  are  included  in  Table  1-1.  The  mean  time  to  turn  was  similar  to  the  values  from  the  first 
B727  study.  However,  the  longest  response  time  was  1 1  seconds.  The  conclusion  was  that 
improving  the  situational  awareness  of  the  pilots  to  the  meaning  of  the  word  "immediate  did  not 
affect  the  general  response  characteristics  for  the  subject  populations,  but  it  did  remove  the 
unwanted  long  responses.  A  recommendation  for  safety  improvement  was. 

...to  include  the  meaning  of  the  word  "immediately"  or  another  word  that 
conveys  this  concept  of  urgency  as  it  applies  to  ATC  direction  to  pilots  for 
aircraft  maneuvers  in  all  phases  of  training,  and  specifically,  include  ATC 
directed  missed  approaches  during  recurrent  simulator  training  [8]. 

1.1.3  PRM  Demonstration  Program 

The  purpose  of  the  Precision  Runway  Monitor  (PRM)  Demonstration  Program  was  to 
evaluate  the  use  of  new  radar  and  display  technology  together  with  the  monitor  controller 
position  for  the  safe  conduct  of  simultaneous  parallel  ILS  approaches  to  runways  spaced  between 
3400  and  4300  feet  apart.  Operational  issues  associated  with  conducting  simultaneous 
approaches  to  runways  separated  by  less  than  4300  feet  were  also  considered. 

The  primary  goal  was  to  maintain  current  safety  levels  during  blunder  events.  To  assess 
the  safety  of  the  PRM  system,  a  Monte  Carlo  simulation  of  the  events  depicted  in  Figure  1-1  was 
developed.  Field  and/or  experimental  data  were  collected  for  each  of  the  blunder  parameters: 
aircraft  geometry;  radar  and  blunder  alert  performance;  controller  response  time;  communication 
delays;  and,  performance  of  the  threatened  (evader)  aircraft. 

Data  from  the  B727  and  DC  10  performance  studies  described  in  Section  1.1.2  were 
incorporated  into  the  Monte  Carlo  simulation  as  representative  threatened-aircraft  breakouts. 
The  distributions  of  times  to  start  of  turn  are  depicted  in  Figure  1-2.  One  risk  analysis  included 
all  responses  from  the  DCIO  and  first  B727  studies.  The  results  indicated  an  unacceptable  level 
of  safety  for  the  operation.  A  second  analysis  was  based  on  the  conclusions  of  the  second  B727 
study,  which  indicated  that  increased  pilot  awareness  could  improve  breakout  performance.  For 
this  second  evaluation,  all  breakout  maneuvers  with  a  time  to  start  of  turn  greater  than 
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20  seconds  were  removed  from  the  data  set.  As  illustrated  in  Figure  1-2,  this  resulted  in  a 
distribution  of  responses  with  a  maximum  time  to  start  of  turn  of  17  seconds.  Limiting  evader 
turn  responses  to  17  seconds  or  less  resulted  in  an  acceptable  level  of  safety  [1]. 


TIME  TO  START  OF  TURN  (seconds) 
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Figure  1-2.  B727  and  DCIO  data  used  in  the  PRM  risk  analysis.  Values  are  times  from  start 
of ATC  breakout  instruction  to  start  of  left-hand  roll,  (a)  1 00-foot  DA  and  2 00-foot  DA, 

combined;  (b)  1800 feet  AGL. 


During  one  of  the  PRM  Demonstration  Program  simulations,  a  McDonnell-Douglas  80 
(MD80)  pilot  had  difficulty  disconnecting  the  autoland  in  a  MD80  cockpit  simulator  in  order  to 
fly  a  manual  breakout.  Since  the  MD80  has  a  more  modem  avionics  system  than  the  B727  and 
DCIO,  this  event  raised  the  issue  of  possible  difficulty  executing  an  ATC-directed  breakout  by 
newer  aircraft  types  such  as  the  B747-400,  Boeing  757,  and  Boeing  767  when  the  approach  is 
flown  using  the  autopilot. 

The  PRM  Program  concluded  in  1991  with  documentation  of  the  issues,  the  supporting 
data  and  results,  and  the  following  recommendation: 

It  is  recommended  that  the  FAA  issue  a  national  standard  for  runway 
spacing  of  3,400  feet,  provided  the  approaches  can  be  monitored  by 
displays  equivalent  to  those  used  in  the  demonstration,  driven  by  a  radar 
accurate  to  within  1  milliradian  with  an  update  interval  of  2.4  seconds  or 
less. ...  A  familiarization  program  to  ensure  that  all  pilots  understand  their 
responsibilities  during  a  closely  spaced  parallel  approach  will  also  be 
required.  An  off-centerline  obstmction  evaluation  will  be  conducted  at  all 
airports  where  PRM  is  to  be  installed  [1]. 
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1.1.4  MPAP  Simulations 

The  Multiple  Parallel  Approach  Program  (MPAP)  started  in  1988  with  real-time 
simulations  of  proposed  multiple  parallel  approach  procedures  for  Dallas-Fort  Worth 
International  Airport  [3].  Since  then,  the  sophistication  of  the  simulation  facility  at  the  FAA 
Technical  Center  has  increased,  with  improvements  in  the  design  of  test  cases  (blunders)  and 
incorporation  of  live  modem  feeds  to  remote-site  cockpit  simulators.  Currently,  the  facility  has 
the  capacity  to  incorporate  up  to  seven  remote  simulator  sites. 

One  of  the  first  MPAP  simulations  to  incorporate  multiple  simulators  was  the  study  of 
dual  parallel  ILS  approaches  to  runways  spaced  3000  feet  apart,  conducted  in  1994  [9].  This 
study  used  Boeing  747-400,  Lockheed  1011,  Boeing  727,  and  Boeing  737  cockpit  simulators  as 
threatened  aircraft  during  blunder  scenarios  in  order  to  provide  realistic  aircrew  and  aircraft 
response  characteristics.  Although  some  of  the  test  acceptance  criteria  were  met,  the  target 
safety  level  criterion  was  not  met.  One  of  the  factors  identified  with  poor  blunder  resolution 
performance  was  inadequate  breakout  maneuvering.  The  MPAP  technical  working  group 
concluded  that  the  pilots  were  unfamiliar  with  ATC-directed  breakouts  between  glide  slope 
intercept  and  decision  altitude.  Lack  of  familiarization  was  compounded  by  the  specific  cockpit 
procedures  required  to  elicit  a  breakout  in  aircraft  with  highly-automated  flight  control 
management  systems  during  autopilot-coupled  approaches. 

The  MPAP  technical  working  group  decided  that  the  effectiveness  of  improving  pilot 
awareness  as  well  as  improving  cockpit  breakout  technique  needed  to  be  evaluated  for  advanced 
automation  aircraft  such  as  the  B747-400.  The  group  requested  that  its  research  be  incorporated 
into  the  A320  and  B747-400  test  plans.  The  group  then  used  the  results  of  the  two  stupes  during 
the  development  of  pilot  training  programs  for  subsequent  simulations  conducted  in  October 
1995  [9]  and  April  1996  [10].  Both  of  these  simulations  were  successful;  and  improved  pilot 
training  was  mentioned  as  a  contributing  factor. 

1.2  STUDY  OBJECTIVES 

Because  of  the  difficulty  observed  with  the  MD80  automation  during  the  PRM 
Demonstration  Program,  the  current  study  was  commissioned  to  measure  aircrew  and  aircraft 
breakout  performance  for  newer  aircraft  types,  given  current  procedures.  Because  of  the  slow 
responses  observed  during  the  MPAP  simulation  in  1994,  this  study  was  expanded  to  include 
evaluation  of  the  effect  of  pilot  training  on  breakout  performance. 

The  primary  objective  of  the  A320  study  was  to  determine  if  pilot  training  improved 
A320  breakout  response.  Phase  1  of  the  study  was  used  as  a  baseline  to  measure  A320  breakout 
performance  given  current  procedures  and  aircrew  training.  Phase  2  tested  the  effect  of  a 
cockpit-specific  breakout  procedure  combined  with  pilot  situational  awareness  training  and 
modified  controller  phraseology  on  the  breakout  maneuver. 

The  measure  used  to  determine  acceptable  performance  was  time  from  start  of  the  ATC 
breakout  instruction  to  start  of  roll.  Although  a  breakout  maneuver  involves  pitch,  thrust,  and 
roll  changes,  time  to  start  of  roll  was  identified  as  the  critical  measure  of  performance.  The 
B727  and  DCIO  breakout  performance  data  that  resulted  in  the  successful  PRM  risk  analysis 
were  used  as  the  criteria  for  acceptable  breakout  performance.  As  mentioned  in  Section  1.1.3, 
the  PRM  risk  analysis  indicated  an  acceptable  level  of  safety  when  the  breakout  maneuvers  with 
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times  to  start  of  turn  greater  than  20  seconds  were  removed.  The  resulting  distributions  of  times 
to  start  of  turn  had  mean  values  of  7  to  8  seconds,  and  maximum  values  of  17  seconds.  These 
statistics  were  used  as  test  criteria  for  the  current  study. 

The  secondary  objective  of  the  study  was  to  evaluate  factors  that  can  affect  A320 
breakout  performance.  The  tested  factors  included  the  use  of  autopilot,  the  altitude  at  which  the 
breakout  was  executed,  and  the  vertical  direction  of  the  breakout  (climb  or  descend). 
Interactions  among  factors  were  also  evaluated. 

To  summarize,  the  testable  questions  were: 

Phase  1:  What  is  A320  breakout  performance  given  current  ATC  breakout 
phraseology  and  airline  procedures?  How  does  this  performance  compare 
to  the  B727  and  DC  10  response  distributions  used  in  the  PRM  risk 
analysis? 

Phase  2:  Does  increased  situational  awareness  combined  with  an  aircraft-specific 
breakout  procedure  improve  A320  breakout  performance  by  reducing  the 
time  to  start  of  maneuver?  How  does  this  performance  compare  to  the 
B727  and  DC  10  response  distributions  used  in  the  PRM  risk  analysis? 

1.3  REPORT  TERMINOLOGY 

While  the  jargon  used  in  this  report  is  based  on  the  jargon  used  by  line  pilots,  there  may 
be  some  differences  in  the  application  of  the  phrases.  In  order  to  avoid  confusion,  the  context  of 
each  term,  as  used  in  the  remainder  of  this  report,  is  defined  below. 

Breakout  (BO):  An  ATC-directed  deviation  away  from  the  final  approach  course  in 
response  to  the  actions  of  another  aircraft  on  the  adjacent  parallel  approach.  The  instruction 
includes  a  new  heading,  turn  direction  (left  or  right),  new  altitude,  and  vertical  direction  (climb 
or  descend).  In  air  traffic  jargon,  this  may  also  be  called  a  go-around. 

Approach  Mode:  The  flight  director  and  autopilot  configuration  during  final  approach. 
HE  mode  indicates  the  flight  director  is  on,  the  autopilot  is  off,  and  the  pilot  is  flying  manually, 
using  the  flight  director  for  guidance.  AP  mode  indicates  that  both  the  flight  director  and 
autopilot  are  on,  and  the  flight  management  system  is  controlling  the  aircraft.  In  this  report,  HF 
mode  is  referred  to  as  a  hand-flown,  or  manual,  approach,  and  AP  mode  is  referred  to  as  an 
autopilot-coupled  approach. 

Breakout  Mode;  The  autopilot  configuration  at  the  start  of  the  turn.  HF  mode  indicates 
that  the  autopilot  is  off  at  the  start  of  the  turn;  the  flight  director  may  be  on  or  off.  AP  mode 
indicates  the  autopilot  is  on  at  the  start  of  the  turn.  This  distinction  is  made  because  some 
subjects  disengaged  the  autopilot  after  the  start  of  turn.  In  this  report,  HF  mode  is  referred  to  as 
a  hand-flown,  or  manual,  breakout  and  AP  mode  is  referred  to  as  an  autopilot-coupled  breakout. 

Approach/Breakout  (Appr/BOl  pair;  HF/HF  indicates  a  hand-flown  approach 
followed  by  a  hand-flown  breakout,  even  if  the  crew  turns  on  the  autopilot  after  the  start  of  the 
breakout  maneuver.  This  distinction  is  made  because  some  crews  engaged  the  autopilot  once 
they  had  achieved  the  breakout  heading  and  altitude.  AP/HF  indicates  an  autopilot-coupled 
approach  followed  by  a  manual  breakout,  meaning  the  autopilot  was  disconnected  at  some  time 
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after  the  start  of  the  ATC  breakout  instruction  and  before  the  start  of  turn.  AP/AP  indicates  an 
autopilot-coupled  approach  followed  by  an  autopilot-coupled  breakout. 

Start  of  Turn;  Data  record  at  which  the  aircraft  achieved  a  3-degree  or  greater  roll  to  the 
left  and  maintained  the  roll  until  the  final  heading  was  reached. 

Start  of  Climb:  Data  record  at  which  descent  was  arrested  and  vertical  rate  increased  at 
least  150  feet  per  minute  above  the  descent  rate  at  the  start  of  the  ATC  transmission. 
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2.  EXPERIMENTAL  DESIGN 


This  study  was  designed  in  two  steps.  First,  Phase  1  was  designed  to  test  A320  breakout 
performance  given  current  procedures.  Then,  Phase  2  was  designed  to  test  the  effect  of  proposed 
pilot  training  and  ATC  breakout  phraseology  on  A320  breakout  performance.  The  experimental 
designs  for  Phase  2  was  dependent  on  the  knowledge  gained  from  the  previous  phase  and  from 
the  B747-400  study.  Because  of  the  dependency  of  the  second  phase  on  previous  evaluations 
and  because  of  the  time  frame  available  for  initial  test  development,  the  two  phases  were  not 
designed  concurrently.  This  resulted  in  variations  in  the  test  scenarios  and  in  the  pilot  surveys 
used  in  each  of  the  two  phases.  The  following  sub-sections  describe  the  experimental  design 
developed  for  each  phase  of  the  study. 

2.1  PHASE  1 

Phase  1  was  designed  to  be  comparable  to  the  B727  and  DCIO  studies  in  order  tg 
facilitate  comparison.  In  addition,  Phase  1  used  the  same  design  as  used  in  the  first  phase  of  the 
B747-400  study.  Since  these  were  the  first  studies  specifically  designed  to  evaluate  the 
performance  of  an  advanced  avionics  aircraft  during  the  final  approach  phase,  the  test  scenarios 
included  missed  approaches  and  ATC-directed  breakouts  at  decision  altitude.  The  value  of  the 
breakouts  at  decision  altitude  was  two-fold.  First,  since  the  breakouts  were  executed  close  to  the 
ground,  the  pilots  had  to  achieve  a  positive  climb  rate  before  the  turn  could  be  executed.  This 
provided  an  upper  bound  on  the  time  required  to  turn  away  from  the  approach.  Second,  since  the 
pilots  would  start  the  turn  earlier  than  they  would  for  a  missed  approach,  the  low-altitude 
breakouts  also  provided  a  bound  for  the  required  obstacle  assessment  surface.  Although 
included  as  a  design  consideration,  the  obstacle  clearance  evaluation  is  not  reported  in  this 
document. 

The  Phase  1  design  included  breakouts  at  decision  altitude  and  at  1800  feet  above  ground 
level  (AGL);  the  same  as  in  the  previous  studies.  Scenarios  at  500  feet  AGL  were  added  so  the 
pilots  would  not  learn  to  expect  breakouts  at  only  two  altitudes.  The  500-foot  scenarios  also 
provided  information  about  breakout  performance  near  the  automation  transition  altitude  of 
400  feet  AGL*.  In  order  to  satisfy  user-community  concerns  about  the  realism  of  the  breakout 
scenarios,  the  distraction  scenarios  were  retained.  Due  to  time  constraints,  an  aircraft-specific 
distraction  could  not  be  developed,  so  the  right  engine  failure  distraction  used  in  the  B727  and 
DCIO  studies  was  retained.  Finally,  two  scenarios  were  added  in  which  the  air  traffic  controller 
directed  the  aircraft  to  descend  rather  than  climb. 

2.1.1  Independent  Variables 

In  evaluating  evader  breakout  performance  given  current  procedures,  there  were  four 
independent  variables:  approach  mode,  level  of  distraction,  altitude  at  start  of  breakout,  and 
vertical  direction  of  the  breakout.  The  levels  for  each  variable  are  listed  in  Table  2-1.  The 
design  was  treated  as  a  two-factor  random  model,  with  approach  mode  combined  with  each  of 


The  A320  automation  does  not  accept  manual  heading  input  when  the  aircraft  is  below  400  feet  radio  altitude. 
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the  other  three  variables:  approach  mode  with  level  of  distraction;  approach  mode  with  altitude 
of  breakout;  and,  approach  mode  with  direction  of  breakout. 


Table  2-1.  Independent  Variables  For  Phase  1 


Independent  Variable 

Test  Levels 

Control  Variables 

Approach  Mode 

1.  Hand-flown  using  Flight  Director  (HF) 

2.  Autopilot-coupled  (AP) 

N/A 

Altitude 

1 .  Decision  Altitude  (DA) 

2.  500  feet  AGL  (500‘) 

3.  1800  feet  AGL  (1800') 

Distraction  (None) 
Direction  (Climb) 

Direction 

1,  Climb 

2.  Descend 

Distraction  (None) 
Altitude  (1800’) 

Distraction 

1.  None 

2.  Engine  Out 

Altitude  (DA) 
Direction  (Climb) 

For  test  sequence  development,  approach  mode  was  treated  as  one  test  condition,  and  the 
other  three  variables  were  combined  under  the  condition  "breakout  group."  The  number  of 
replicates  assigned  to  each  pair  of  conditions  is  listed  in  Table  2-2.  The  reported  weather  for  all 
hand-flown  approaches  was  at  Category  I  weather  minimum  (200-foot  ceiling),  while  the 
reported  weather  for  all  autopilot-coupled  approaches  was  at  Category  U  weather  minimum 
(100-foot  ceiling).  The  crews  were  instructed  to  follow  company  procedure  where  applica  e. 
Otherwise,  they  were  instructed  to  alternate  the  pilot  flying  the  approach  (captain  or  first  officer). 
Because  the  crews  were  to  follow  company  procedures,  the  scenarios  could  not  be  assigned  to 
each  subiect  a  priori.  This  was  because  airline  company  policy  may  dictate  who  is  required  to 
act  as  pilot  flying  under  certain  conditions  such  as  approach  category.  Thus  it  was  not  possible 
to  design  repeated  measurements  for  each  subject  pilot.  The  same  scenario  sequence  was  used 
for  all  subject  crews.  The  sequence  and  test  conditions  for  each  scenario  are  given  m 

Appendix  A. 


Table  2-2.  Number  of  Replicates  for  Phase  1  Scenarios 


Breakout  Group 

Approach  Mode 

DA 

500' 

1800' 

Descend 

Distraction 

hand-flown 

2 

2 

2 

1 

1 

autopilot 

2 

2  ^ 

2 

1 

1 

2.2  PHASE  2 


The  purpose  of  Phase  2  was  to  evaluate  the  effectiveness  of  situational  awareness 
training,  proposed  ATC  phraseology,  and  an  A320-specif.c  breakout  procednre  on  bre^out 
performance.  The  training  package  used  during  the  study  is  described  m  Section  3.3  and 
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Appendix  D.  A  secondary  design  consideration  was  the  estimation  of  practice  effects;  i.e.,  did  a 
subject's  performance  change  as  the  test  session  progressed. 

The  design  was  based  on  the  same  two  conditions  as  in  Phase  1:  breakout  group  and 
approach  mode.  Climbing  breakouts  at  1000  feet-AGL  (1000')  were  added  to  reduce  subject 
anticipation  of  when  the  breakout  would  occur  and  to  provide  data  for  use  in  future  risk 
assessments.  The  distraction  scenarios  were  removed  from  the  design. 

2.2.1  Independent  Variables 

In  evaluating  the  effect  of  the  training,  there  were  three  independent  variables:  approach 
mode,  altitude  at  start  of  breakout,  and  vertical  direction  of  the  breakout.  The  levels  for  each 
variable  are  listed  in  Table  2-3.  The  design  was  treated  as  a  two-factor  within-subject  model, 
with  approach  mode  combined  with  each  of  the  other  two  factors:  approach  mode  with  altitude 
of  bre^out;  and,  approach  mode  with  direction  of  breakout.  ‘ 

Table  2-3.  Independent  Variables  For  Phase  2 


Independent  Variable 

Test  Levels 

Control  Variable 

Approach  Mode 

1.  Hand-flown  using  Flight  Director  (HF) 

2.  Autopilot-coupled  (AP) 

N/A 

Altitude 

1 .  Decision  Altitude  (DA) 

2.  500  feet  AGL  (500') 

3.  1000  feet  AGL  (1000') 

4.  1800  feet  AGL  (1800') 

Direction  (Climb) 

Direction 

1.  Climb 

2.  Descend 

Altitude  (1800’) 

The  reported  weather  for  all  scenarios  except  the  autopilot  breakout  at  decision  altitude 
was  Category  I  weather  minimum.  This  removed  the  airline  policy  constraint  that  was  observed 
in  Phase  1  and  allowed  for  sequence  assignment  by  crewmember:  captain  or  first  officer. 
Assignment  of  scenarios  by  pilot  resulted  in  a  mixed  design:  within  subject  for  altitude  and 
direction,  and  between  subject  for  approach  mode.  Within-subject  replicates  for  the  500  and 
1800'  autopilot-coupled  scenarios  were  used  for  analysis  of  practice  effects.  The  distributions  of 
test  scenarios  for  each  crewmember,  identified  as  subject  A  or  B,  are  listed  in  Table  2-4.  All 
crews  experienced  the  same  set  of  sequenced  scenarios.  One  half  of  the  crews  experienced  the 
set  in  the  forward  order,  while  the  other  crews  experienced  the  set  in  reverse  order.  In  addition, 
crew  assignments  were  varied,  meaning  subject  A  was  the  captain  in  one  half  of  the  crews  and 
subject  A  was  the  first  officer  in  the  other  crews.  This  resulted  in  four  test  sequences.  The 
scenario  order  and  configurations  for  each  test  sequence  are  given  in  Appendix  A. 
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Table  2-4  Number  of  Replicates  for  Phase  2  Scenarios 


1 

Breakout  Grou 

P 

Pilot 

Approach 

- 

Flying 

Mode 

DA 

500’ 

700' 

1800' 

Descend 

A 

hand-ficwn 

1 

1 

- 

1 

- 

autopilot 

- 

2 

1 

2 

1 

B 

hand-flown 

- 

1 

1 

1 

1 

autopilot 

1 

2 

- 

2 

- 

2.3  RESEARCH  QUESTIONS 

Phase  1  of  the  study  was  designed  to  answer  the  following  research  questions: 

1.  Does  A320  breakout  performance,  given  current  procedures,  meet  the  requirements 
established  during  the  PRM  Demonstration  Program  for  mean  and  maximum  values  of 
time  to  start  of  turn? 

2.  Are  pilots  comfortable  with  executing  breakouts  at  low  altitudes  (less  than  400  feet 
AGL)? 

3.  Are  pilots  comfortable  with  executing  descending  breakouts  at  higher  altitudes  (above 
1000  feet  AGL)? 

4.  Do  cockpit  distractions  affect  breakout  performance? 

Phase  2  of  the  study  was  designed  to  answer  the  following  research  questions: 

5.  Does  situational  awareness  training  combined  with  a  self-administered  written  A320 
breakout  procedure  improve  the  mean  and/or  maximum  time  to  start  of  turn  sufficiently 
to  meet  the  requirements  established  during  the  PRM  Demonstration  Program? 

6.  Which  tools  were  effective  in  increasing  pilot  situational  awareness? 


Finally,  the  study  was  designed  to  answer  the  following  research  questions  for  each 

phase: 

7.  Does  altitude  affect  breakout  performance? 

8.  Does  vertical  direction  of  the  breakout  affect  breakout  performance? 

9.  Does  approach  mode  affect  breakout  performance? 
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3.  METHODS 


3.1  SUBJECTS 

Commercial  airline  pilots  were  recruited  through  notices  to  the  Air  Line  Pilots 
Association  (AT. PA)  and  to  United  States  air  carriers  that  operate  A320  aircraft.  Subject  pilots 
came  from  Northwest  Airlines,  United  Airlines,  and  America  West  Airlines. 

3.2  FACILITY 

The  study  was  conducted  at  the  Northwest  Aerospace  Training  Corporation  (NATCO)  in 
Eagan,  Minnesota.  The  subject  crew  and  all  test  personnel  occupied  the  certified  full-motion 
A320  cockpit  simulator.  An  observer  sat  behind  the  first  officer's  seat.  The  observer’s  duties 
were  to  verify  that  the  simulator  was  properly  configured  for  each  test  run  and  to  record 
observations  during  each  session.  A  Northwest  Airlines  pilot/instructor  operated  the  simulator 
controls  located  behind  the  captain's  seat.  The  simulation  operator’s  duties  were  to  configure  the 
simulator  for  each  test  run,  record  data,  and  help  the  subjects  with  equipment-related  problems. 
A  third  person  sat  next  to  the  simulation  operator  and  read  the  controller  scripts.  All  test 
personnel  who  acted  as  controller  had  previous  air  traffic  control  experience.  All  test  personnel 
were  instructed  not  to  provide  information  to  the  subject  crews  that  might  affect  the  results  or  to 
provide  suggestions  about  cockpit  procedures. 

3.2.1  A320  Flight  Deck  Controls 

Since  the  A320  is  the  first  certificated  commercial  aircraft  with  fly-by- wire  technology, 
the  cockpit  automation  may  be  unfamiliar  to  some  readers.  This  section  describes  the  controls 
and  displays  that  were  important  during  the  breakout  maneuver  and  some  of  the  differences 
between  the  A320  and  conventional  advanced-automation  cockpits  such  as  the  B747-400.  The 
purpose  of  this  section  is  to  provide  the  reader  with  an  understanding  of  the  measured  data  and  to 
provide  insight  needed  for  interpreting  the  results.  This  is  not  intended  to  be  a  primer  on  A320 
operations.  In  order  to  facilitate  the  description  of  the  cockpit  displays,  the  simulator  is  referred 
to  as  the  "aircraft." 

The  first  officer  instrument  panel  and  the  pedestal  of  the  NATCO  A320  cockpit  simulator 
are  shown  in  Figure  3-1.  One  difference  between  the  A320  and  other  commercial  aircraft  is  the 
lack  of  a  control  wheel.  Instead,  the  pilot  uses  the  side  stick  to  control  pitch  and  roll.  The  side 
stick  is  outboard  of  the  seat:  to  the  left  for  the  captain,  and  to  the  right  for  the  first  officer. 
During  automated  flight,  pressing  the  button  on  the  stick  disengages  the  autopilot  and  allows  the 
pilot  to  take  control  of  the  aircraft.  During  manual  flight,  pressing  the  button  overrides  the  other 
stick;  but  this  is  not  a  normal  procedure. 

The  Flight  Control  Unit  (FCU)  is  the  upper  center  section  of  the  instrument  panel,  located 
in  the  upper  left  comer  of  Figure  3-1.  The  Flight  Control  Unit  is  comparable  to  the  Mode 
Control  Panel  in  a  Boeing  aircraft,  and  houses  the  speed,  heading,  altitude,  and  vertical  speed 
knobs.  The  crew  can  enter  a  new  value  for  any  of  these  flight  parameters  either  via  the  FCU 
knob  or  by  entering  the  value  into  the  Multipurpose  Cockpit  Display  Unit  (MCDU)  located  on 
the  pedestal.  During  this  study,  manual  entries  were  made  using  the  FCU  knobs. 
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Figure  3-1.  A320 flight  deck.  This  view  shows  the  central  pedestal  and  the 
first  officer's  side  of  the  instrument  panel 


Instead  of  conventional  throttles,  the  A320  has  thrust  levers  that  are  located  on  the 
pedestal.  Each  thrust  lever  has  six  positions  (four  detents  and  two  limits).  The  forward  limit  is 
TOGA  (Take  Off/Go  Around).  Next  is  the  FLX/MCT  (Flexible  Reduced  Thrust/  Maximum 
Continuous  Thrust)  detent,  followed  by  the  CL  (climb).  Forward  Idle,  and  Reverse  Idle  detents. 
The  aft  limit  of  the  thrust  levers  is  Maximum  Reverse. 

Auto  thrust  is  engaged  under  normal  operations  and  is  typically  employed  by  the  pilot 
flying  as  follows.  The  A320  pilot  selects  either  TOGA  or  FLX/MCT  for  take-off.  Thrust  is 
reduced  to  CL  at  1500  feet  AGL,  the  default  thmst  reduction  altitude,  and  the  aircraft  operates  at 
CL  until  just  prior  to  landing.  The  pilot  selects  Forward  Idle  just  prior  to  touch  down.  Once  on 
the  runway.  Reverse  Idle  and  Max  Reverse  are  used  to  decrease  airspeed.  Although  autothrust  is 
used  during  normal  operations,  the  pilot  flying  has  the  option  of  disconnecting  autothrust  and 
using  the  thrust  levers  as  conventional  throttles  during  non-standard  procedures. 

The  two  thrust  lever  positions  used  during  this  study  were  CL  and  TOGA.  At  the 
beginning  of  each  trial,  the  thrust  levers  were  set  to  CL  and  autothrust  was  engaged.  During 
climbing  breakouts,  a  positive  rate  of  climb  was  achieved  by  using  either  of  two  procedures. 

1)  The  pilot  flying  moved  the  thrust  levers  to  TOGA  and  initiated  an  immediate 
go-around  maneuver  using  the  maximum  available  rated  thrust.  Aircraft  pitch 
attitude  immediately  increased  from  approximately  +3  degrees  to  a  nominal 
+18  degrees. 

2)  The  pilot  flying  left  the  thrust  levers  in  the  CL  detent.  “Open  Climb”  mode 
was  selected  by  setting  the  altitude  to  a  level  higher  than  the  existing  aircraft 
altitude  displayed  in  the  FCU  altitude  window  and  pulling  the  altitude  select 
knob.  The  engines  smoothly  advanced  to  climb  thrust  and  the  autopilot  and/or 
flight  directors  commanded  an  increase  in  pitch  attitude  to  maintain  existing 
airspeed.  While  less  dramatic  than  a  go-around  maneuver.  Open  Climb 
arrested  the  sink  rate  without  an  abrupt  change  in  pitch  attitude. 

The  Flight  Management  System  (FMS)  operates  under  managed  guidance  or  under 
selected  guidance.  In  managed  guidance,  the  speed,  heading,  altitude,  and  vertical  speed  in  the 
flight  director  and  autopilot  are  based  on  the  flight  plan  entered  into  the  FMS  via  the  MCDU.  In 
selected  guidance,  the  value  for  a  given  flight  parameter  is  manually  selected  by  a  crewmember. 
The  guidance  mode  for  each  flight  parameter  depends  on  the  position  of  the  respective  FCU 
knob.  When  a  knob  is  depressed  (pushed),  that  control  value  is  based  on  managed  guidance  and, 
except  for  altitude,  the  window  above  the  knob  displays  three  dashes.  When  the  knob  is  pulled, 
the  pilot  overrides  the  managed-guidance  value  and  the  selected  value  is  displayed  in  the 
corresponding  window.  For  example,  during  normal  operations  the  crew  may  need  to  override 
speed  control  if  air  traffic  control  requests  a  speed  change.  During  this  study,  the  aircrews  used 
selected  guidance  for  heading  and  altitude,  if  needed,  during  the  ATC-directed  breakouts. 

In  Figure  3-1,  dashes  are  displayed  in  the  speed  and  vertical  speed  windows,  indicating 
that  these  values  were  based  upon  managed  guidance.  The  heading  window  in  Figure  3-1 
displays  a  value,  indicating  that  the  heading  knob  was  pulled  and  managed  navigation  was 
overridden.  Note  that  the  FCU  always  has  an  altitude  value  displayed,  even  in  managed  flight. 

The  flight  control  system  provides  limits  on  pitch,  roll,  speed,  et  cetera,  to  prevent  the 
pilot  flying  from  inadvertently  exceeding  the  aircraft  limitations.  There  are  no  company-induced 
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limits  within  the  flight  management  system;  all  values  are  set  by  the  manufacturer.  The  flight 
control  system  dynamically  calculates  each  limit  based  primarily  on  aircraft  configuration. 
Depending  on  the  status  of  the  primary  flight  control  computers,  the  flight  control  systein  will 
operate  under  Normal  Law  or  under  various  other  control  laws  as  the  system  degrades.  It  is  not 
possible  for  the  pilot  to  override  any  flight  control  limit  during  manual  flight  when  the  aircraft  is 
operating  under  Normal  Law.  However,  these  limits  are  much  larger  than  the  operating 
parameters  for  pitch  and  roll  angles,  and  should  not  be  confused  with  the  Federal  Aviation 
Regulation  (FAR)  limits.  As  will  be  described  in  the  results  section,  some  pilots  exceeded  the 
FAR  roll  limit  of  30  degrees,  yet  remained  within  the  flight  control  system’s  limit. 

For  this  study,  the  final  approaches  were  flown  either  manually  with  only  the  flight 
director  and  autothrust  engaged,  or  automatically  with  the  autopilot  engaged.  During  each 
breakout,  the  pilot  flying  had  the  option  of  flying  manually  or  of  using  the  autopilot.  Following 
manual  approaches,  all  pilots  flew  manual  breakouts.  Following  autopilot-coupled  approaches, 
the  pilot  flying  could  disconnect  the  autopilot  by  pushing  the  button  on  the  side  stick,  then  fly  a 
manual  breakout.  If  the  pilot  flew  the  breakout  using  the  autopilot,  the  new  heading  and  altitude 
were  selected  in  the  Flight  Control  Unit.  The  new  flight  path  was  then  displayed  on  the 
Navigation  Display  and  the  new  flight  parameters  were  displayed  on  the  Primary  Flight  Display. 
Entering  the  new  heading  and  altitude  information  into  the  FCU  was  not  as  critical  for  manual 
breakouts  because  the  flight  director  was  used  only  as  a  reference,  if  at  all,  and  did  not  control 
the  flight  path. 

If  the  thrust  levers  are  at  the  CL  detent  when  the  new  altitude  is  entered,  pitch  and  thrust 
are  based  on  open  climb.  If  the  thrust  levers  are  advanced  to  TOGA,  the  flight  directors  and 
autopilot  enter  the  Go-Around  mode  as  follows.  As  soon  as  the  thrust  levers  are  set  to  the 
TOGA  stop,  the  go-around  mode  is  engaged.  Lateral  guidance  is  now  in  GA  TRK  (go-around 
track).  For  safety  reasons,  no  action  may  be  performed  on  the  heading  knob  until  the  aircraft  is 
above  400  feet  radio  altitude.  Above  400  feet,  pushing  the  heading  knob  disengages  GA  TRK 
and  places  the  system  in  Managed  (lateral)  Navigation.  In  contrast,  pulling  the  heading  knob 
disengages  GA  Track  and  places  the  system  in  Selected  (lateral)  Navigation,  i.e.,  in  heading 
navigation. 

The  effect  of  TOGA  on  the  autopilot  breakouts  conducted  during  this  study  was  two-fold. 
First,  the  flight  director  would  not  accept  the  manually-selected  heading  until  the  aircraft  was 
above  400  feet.  Second,  if  the  thrust  levers  were  set  to  the  TOGA  position  after  the  breakout 
heading  had  been  selected  and  while  the  autopilot  was  engaged,  the  aircraft  turned  back  towards 
GA  Track  (ground  track  at  the  time  of  TOGA  selection).  The  crew  then  had  to  override  the 
flight  management  system  or  disconnect  the  autopilot  and  fly  a  manual  breakout.  The  effect  of 
TOGA  on  descending  as  well  as  climbing  breakouts  was  a  consideration  in  the  development  of 
the  A320  breakout  procedure  tested  during  Phase  2. 

3.3  TEST  PROCEDURES 

Phase  1  was  conducted  during  February  1995,  and  Phase  2  was  conducted  during 
June  1995.  Upon  arrival  at  the  test  facility,  each  subject  crew  was  sent  to  a  briefing  room.  Test 
personnel  used  a  briefing  script  to  inform  the  subjects  that  they  would  be  flying  approaches  to 
Memphis  International  Airport  (MEM)  runway  36L.  The  approach  plate  to  be  used  was  given  to 
each  crewmember.  For  Phase  1,  the  current  Category  I  and  Category  II  NOS  approach  plates 
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were  used.  For  Phases  2,  the  Jeppeson  approach  plates  for  Memphis  were  modified  to  include 
information  about  close  parallel  approaches.  The  Phase  1  approach  plates  are  presented  in 
Appendix  B,  and  the  modified  approach  plates  are  in  Appendix  C. 

In  Phase  1,  the  subjects  were  told  to  follow  company  procedures  and  to  follow  air  traffic 
control  instructions  when  given.  In  Phases  2,  the  subjects  received  training  material  to  review 
before  the  test  session.  The  training  materials  are  described  in  Sections  3.3.2  and  3.3.3. 

Each  crew  was  given  the  opportunity  to  fly  practice  approaches  before  the  test  session  if 
the  subjects  were  not  familiar  with  the  configuration  of  the  NATCO  cockpit  simulator.  Each  test 
session  lasted  approximately  three  hours.  There  was  a  half-hour  break  mid-way  through  the  test 
sequence.  After  the  test  session,  the  subjects  completed  a  pilot  survey.  The  survey  for  each 
phase  of  the  study  was  designed  based  on  the  research  questions  for  that  phase. 

3.3.1  Flight  Procedures 

By  design,  the  simulator  maintained  a  gross  weight  of  141,900  pounds  during  the  entire 
final  approach.  Before  each  trial,  the  simulator  was  set  to  intercept  altitude  at  8  nautical  miles 
from  the  threshold.  Landing  gear  and  flaps  were  set  according  to  company  procedure.  Altimeter 
was  set  to  29.92"  Hg.  The  captain  and  first  officer  both  flew  the  aircraft.  In  Phase  1,  the  pilot 
flying  was  based  on  company  policy  where  it  existed.  Otherwise,  the  crewmembers  alternated 
who  flew  the  aircraft.  In  Phases  2,  the  pilot  flying  was  based  on  the  test  sequence.  The 
approaches  was  flown  either  manually,  with  the  flight  director  on,  or  autopilot-coupled  according 
to  the  test  sequence. 

3.3.2  Situational  Awareness  Training 

In  Phase  2,  the  subjects  reviewed  a  situational  awareness-training  package  during  the 
briefing.  The  training  package  consisted  of  a  short  video  describing  close  parallel  approach 
operations  [11],  an  11-0  Information  Page  describing  close  parallel  procedures,  an  awareness¬ 
training  bulletin,  and  a  self-administered  multiple-choice  test.  The  11-0  Information  Page  and 
the  bulletin  emphasized  the  need  for  prompt  compliance  when  instructed  to  break  out  of  the 
approach  by  the  monitor  controller.  The  test  was  open-book  and  intended  to  reinforce  the  ideas 
presented  in  the  video  and  written  material.  The  written  training  materials  are  presented  in 
Appendix  D. 


3.3.3  Cockpit  Procedure  Training 

After  the  subjects  reviewed  the  situational  awareness  package,  they  received  a  procedure¬ 
training  package.  The  pilots  read  the  1-page  breakout  procedure  then  took  an  open-book  test. 
The  purpose  of  the  test  was  to  reinforce  the  concepts  in  the  procedure-training  bulletin.  This  was 
the  only  procedure  training  that  the  pilots  received.  They  were  not  given  any  instructions  by  any 
test  personnel  nor  did  they  practice  the  procedure  before  the  test  session. 

For  breakouts  below  400  feet  above  ground  level  (AGL),  the  crew  was  to  fly  the  normal 
missed  approach  procedure  and  select  the  heading  and  altitude  given  in  the  ATC  breakout 
instruction.  For  breakouts  above  400  feet  AGL,  the  crew  changed  heading  and  altitude  to 
selected  guidance  and  configured  the  aircraft  appropriately.  The  procedure  mentioned  using 
“open  climb”  for  climbing  breakouts,  but  did  not  recommend  against  using  TOGA  to  achieve 
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positive  climb.  For  descending  breakouts,  the  crew  was  to  continue  the  descending  vertical 
speed.  The  procedure  bulletin  and  test  are  presented  in  Appendix  D. 

3.3.4  Air  Traffic  Control  Phraseology 

The  air  traffic  control  breakout  phraseology  used  in  Phase  1  was  taken  from  FAA  Order 
7110.65H[12]: 

(Aircraft)  TURN  (left/right)  IMMEDIATELY  HEADING  (degrees), 

CLIMB  AND  MAINTAIN  (altitude). 

The  air  traffic  control  breakout  phraseology  used  in  Phases  2  included  additional 
information  at  the  beginning: 

(Aircraft)  TRAFHC  ALERT.  (Aircraft)  TURN  (left/right) 

IMMEDIATELY  HEADING  (degrees),  CLIMB  AND  MAINTAIN 
(altitude). 

The  addition  of  "traffic  alert"  at  the  beginning  was  recommended  by  a  sub-group  of  the  Multiple 
Parallel  Approach  Procedure  technical  working  group.  It  was  felt  that  the  phrase  would  alert  all 
aircrews  using  the  local  controller  frequency  to  expect  transmission  of  an  emergency  procedure. 
In  addition,  the  additional  text  would  reduce  the  chance  of  a  clipped  or  blocked  breakout 
instruction. 

3.4  DATA  COLLECTION 

Aircraft  metric  data  from  the  A320  simulator  were  collected  by  a  program  running  on  the 
Gould  computer  controlling  the  A320  simulator  at  NATCO.  NATCO  personnel  transferred  the 
data  from  the  Gould  disks  to  9-track  magnetic  tapes  and  mailed  them  to  Lincoln  Laboratory. 
Data  were  extracted  from  the  tapes  using  a  custom  program  on  a  VAX  computer  under 
OpenVMS. 

The  basic  format  of  the  data  files  was  a  header  with  time  and  date  information  followed  by 
some  number  of  data  records  with  aircraft  metric  data  for  each  track.  Data  records  were  written  at 
a  rate  of  3  Hertz  and  3.75  Hertz  for  Phases  1  and  2,  respectively.  Each  record  included  a  time 
stamp,  information  about  test  settings  and  the  aircraft  position,  orientation,  and  configuration. 

Aircraft  position  and  orientation  data  included  latitude  and  longitude,  altitude  and  climb 
rate,  heading,  bank  angle,  pitch  angle,  and  indicated  airspeed.  Aircraft  configuration  data 
included  engine  rpm  (both  engines),  TOGA  indicator,  and  autopilot  indicator.  Test  settings 
included  wind  direction  and  speed,  the  ATC  event  marker  which  indicated  when  the  ATC 
pressed  the  marker  button,  and  a  test  identification  string  which  encoded  scenario  number,  date, 
and  trial  number. 

In  addition  to  the  computer  data,  videotape  recordings  of  the  cockpit  were  made  during 
testing  using  in-cockpit  video  recording  equipment  normally  used  in  pilot  training.  The  camera 
was  located  behind  and  above  the  first  officer,  near  the  center  of  the  aisle.  The  image  frame 
included  the  instrament  panel,  thrust  levers,  and  both  pilots. 
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3.5  DATA  EXTRACTION 


Data  used  in  the  analyses  were  extracted  from  both  the  digital  and  video  data  recordings. 
Data  extraction  from  the  videotapes  was  manual,  while  digital  data  extraction  was  automated. 
The  software  was  written  in  C  and  run  under  Open  VMS  on  VAX  computers,  and  used  the  CA- 
Disspla  graphics  library  for  plotting.  Computed  results  from  both  sources  were  transferred  to 
Macintosh  computers  where  the  analysis  was  performed. 

3.5.1  Digital  Data 

The  data  extraction  software  calculated  time-to-event  values  for  the  test  variables  of 
interest.  The  results  for  each  track  were  saved  in  files  that  were  imported  to  ExceF’^ 
spreadsheets  for  review  and  analysis. 

The  program  first  read  an  entire  track  file  into  memory  and  located  the  ATC  event  mark 
record,  which  was  the  first  record  with  an  event  marker  status  of  “1”  and  signified  the  time  at 
which  the  breakout  instruction  started.  The  records  at  which  events  of  interest  occurred  were 
then  identified  and  used  to  compute  the  time-to-event  values  and  the  altitude  and  distance  firom 
the  threshold  of  the  aircraft  at  each  of  the  events. 

The  calculated  test  variables  are  listed  in  Table  3-1.  Appendix  E  provides  detailed 
algorithms  and  examples  for  each  variable.  The  algorithm  for  identifying  the  start  of  the  turn 
was  the  most  complicated.  The  event  was  nominally  the  point  at  which  the  aircraft  had  rolled  at 
least  three  degrees  to  the  left  (the  aircraft  was  on  approach  to  MEM  36L),  a  criterion  consistent 
with  previous  studies.  However,  it  was  possible  for  the  aircraft  to  roll  back  to  level  or  towards 
the  right.  Since  such  an  action  would  delay  the  start  of  the  turn  maneuver,  the  algorithm  also 
identified  the  last  time  that  the  aircraft  rolled  greater  than  3  degrees  to  the  left.  If  the  algonthms 
did  not  return  the  same  time  to  start  of  roU,  the  correct  value  was  manually  selected. 

The  engine,  pitch,  and  vertical  speed  events  were  determined  by  comparing  the  values  of 
those  metrics  at  the  time  of  the  breakout  instruction  with  the  values  in  successive  records,  using 
the  empirically-determined  thresholds  listed  in  Table  3-1.  Note  that  the  relative  change  criteria 
do  not  always  indicate  a  positive  value.  For  example,  if  vertical  speed  was  -850  feet  per  second 
at  the  start  of  the  ATC  transmission,  then  the  start  of  climb  event  was  the  record  at  which  the 
vertical  speed  was  greater  than  -700  feet  per  second,  indicating  that  the  aircraft  was  still 
descending  but  had  started  accelerating  towards  a  positive  climb  rate. 

3.5.2  Video  Data 

Because  of  the  location  of  the  video  camera,  only  large-scale  movements  by  the  pilots 
were  visible.  The  locations  of  the  controls  mentioned  in  Section  3.2  were  visible,  but  not  the 
exact  settings.  To  illustrate,  the  video  image  in  Figure  3-2  was  recorded  during  one  of  the 
autopilot  breakouts.  The  first  officer  was  the  pilot  flying  and  had  his  hand  on  the  thrust  levers. 
The  captain  was  adjusting  the  heading  knob  on  the  Flight  Control  Unit.  The  bright  lights  at  the 
outer  edges  of  the  Flight  Control  Unit  were  flashing,  indicating  that  the  autopilot  had  been 
disconnected.  The  observer  was  taking  notes.  The  simulator  operator  and  the  controller  (not 
shown)  were  located  to  the  left  and  behind  the  observer. 
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Table  3>1.  Digital  Data  Measurements 


Variable  Name 

Measure 

Criterion 

dt_roll 

Time  to  start  of  aircraft  roll 

Roll  angle  of  at  least  3  deg  to  left,  with 
restrictions 

dt_engine 

Time  to  increase  in  engine  thrust 

Increase  of  500  rpm  over  value  at  marker 
record 

dt_pitch 

Time  to  change  in  pitch 

Increase  of  1 .2  deg  over  value  at  marker 
record 

dt_vertical_speed 

Time  to  change  in  climb  rate 

Increase  of  150  ft/min  over  value  at 
marker  record  (positive  rate  not  required) 

dtjoga 

Time  to  change  in  TOGA  status 

Status  changed  from  off  (0)  to  engaged 
(1) 

maximum_roll 

Greatest  roll  angle  magnitude 

Largest  left-hand  roll  angle  achieved 
during  the  turn  maneuver 

dxjurn 

Distance  traveled  until  start  of  turn 

Difference  between  ground  position  at 
start  of  roll  and  at  start  of  ATC  instruction 

dt_min_altitude 

Time  to  minimum  altitude 

Minimum  radio  altitude 

dz_min_altitude 

Altitude  loss 

Difference  between  altitude  at  start  of  ATC 
transmission  and  minimum  altitude 

*  all  times  are  referenced  to  start  of  ATC  instruction 


The  information  extracted  from  the  videotapes  included  what  pilots  and  testers  did  and 
said.  The  duration  and  exact  phrasing  of  the  ATC  breakout  instruction  was  also  recorded;  and 
any  additional  radio  transmissions  were  noted,  such  as  when  a  pilot  asked  for  part  of  the 
instruction  to  be  repeated.  Also  noted  were  the  pilot  flying;  approach  mode  before  the  breakout 
(autopilot  on  or  off);  and,  the  sequence  of  pilot  actions. 

The  times  from  start  of  ATC  transmission  to  changes  in  autopilot  and  TOGA  status  were 
measured  from  the  video  data.  Change  in  autopilot  status  was  indicated  by  the  flashing  lights 
and  by  an  audible  warning.  Change  to  TOGA  thrust  was  indicated  by  forward  movement  of  the 
thrust  lever.  These  times  were  used  to  validate  the  times  to  event  extracted  from  the  digital  data. 
Agreement  indicated  that  the  ATC  event  marker  was  pressed  at  the  correct  time. 

3.6  DATA  VALIDATION 

The  data  for  each  trial  were  manually  reviewed  before  the  extracted  performance 
variables  were  used.  The  review  focused  on  three  areas:  result  accuracy,  timing  accuracy,  and 
scenario  configuration.  Result  accuracy  was  checked  by  manually  reviewing  the  digital  data  and 
results  for  each  trial.  If  the  automated  process  incorrectly  declared  the  time  of  an  event,  then  the 
time  to  the  event  was  manually  calculated  and  corrected  in  the  ExceF^  spreadsheet. 

Timing  accuracy  was  checked  by  manually  comparing  the  times  to  autopilot  disconnect 
and  to  TOGA  thrust  measured  from  the  videotapes  with  the  values  extracted  from  the  digital 
data.  This  process  revealed  an  error  during  Phase  1  in  the  digital  data  collection  subroutine 
which  incremented  the  time  stamp.  The  error  was  corrected,  and  the  video  and  digital  data 
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Figure  3-2.  Videotape  frame  recorded  during  a  breakout 


results  for  subsequent  test  sessions  agreed.  For  the  sessions  with  incorrect  time  stamps,  the  error 
was  corrected  during  data  extraction  by  incrementing  the  time  data  at  the  correct  rate. 

Scenario  configuration  was  checked  by  manually  verifying  that  the  weather  conditions, 
initial  autopilot  setting,  subject  pilot,  and  breakout  location  agreed  with  the  test  scenario.  In 
addition,  the  breakout  instruction  was  compared  to  the  scripted  text.  If  the  configuration  was 
correct,  the  trial  was  used  in  the  analysis.  If  the  configuration  did  not  match  the  test  scenario, 
then  the  trial  was  retained  only  if  the  configuration  matched  another  test  scenario.  In  this  case, 
the  trial  identification  string  was  changed  to  match  the  assumed  scenario's  identification.  If  the 
trial  configuration  did  not  match  any  of  the  test  scenarios,  then  the  data  for  that  trial  were  not 
used  in  the  analysis. 

In  some  trials,  the  controller  used  the  wrong  call  sign  in  the  breakout  instruction.  During 
Phase  1,  these  events  were  random  and  unintentional.  In  Phase  2,  the  wrong  call  sign  was 
deliberately  used  for  one  breakout  halfway  through  each  session.  The  purpose  of  the  intentional 
error  was  to  check  whether  the  subjects  were  automatically  reacting  without  paying  attention  to 
the  message.  In  each  case,  the  crew  did  not  respond  to  the  breakout  instruction  with  the  incorrect 
call  sign  and  the  controller  had  to  repeat  the  message  with  the  correct  call  sign.  The  data  for 
these  trials  were  not  used  in  the  analysis. 


3.7  DATA  ANALYSIS 

Exploratory  analysis  was  used  to  identify  general  trends  in  breakout  performance.  A 
subset  of  test  variables  representing  turn  and  climb  performance  during  the  breakout  maneuver 
were  selected  for  further  analysis  using  SPSS™,  a  statistical  software  package.  As  described  in 
Section  2,  each  phase  was  designed  as  a  two-factor  model.  The  main  effect  of  each  test  factor  as 
well  as  the  possible  interaction  between  test  factors  were  determined  using  analysis  of  variance 
(ANOVA).  The  null  hypothesis  for  each  analysis  was  that  the  test  means  were  equal.  If  the 
calculated  F  value  was  greater  than  F.o^,  then  the  null  hypothesis  was  rejected  at  the  .05 
significance  level.  If  the  null  hypothesis  was  rejected  for  a  factor  with  more  than  two  levels, 
then  either  the  Tukey’s  b  multiple  range  test  or  paired-sample  t-test  was  used  to  determine  which 
factor  levels  were  statistically  similar.  If  a  subject  experienced  a  given  combination  of  approach 
mode  and  breakout  group  more  than  once,  then  these  repeated  measures  were  averaged  and 
treated  as  a  single  sample.  This  was  done  to  equalize  the  number  of  samples  in  each  test  cell. 

The  ANOVA  formulas  used  depended  on  the  phase.  The  formulas  may  be  found  in  a 
statistical  text  such  as  [13].  For  Phase  1,  the  assignment  of  subjects  to  each  combination  of  test 
factors  was  assumed  to  be  random.  Although  each  subject  experienced  multiple  cases  for  each 
combination  of  approach  mode  and  breakout  group,  there  were  no  complete  sets  of  repeated 
measures.  If  a  repeated-measure  analysis  had  been  used,  then  some  results  would  have  been 
ignored  and  values  for  missing  levels  would  have  been  estimated.  Therefore,  it  was  necessary  to 
assume  a  random,  between-subject  model  for  all  independent  variables.  Collapsing  the  model 
with  respect  to  subject  (i.e.,  use  between-subject  comparisons)  resulted  in  a  more  conservative 
analysis  than  if  the  data  had  been  forced  to  fit  a  within-subject  model. 

A  mixed-factor  model  (within  and  between  subject)  was  used  for  the  Phase  2  analysis. 
Approach  mode  was  the  between-subject  factor.  This  means  that  for  a  given  analysis,  each 
subject  was  assigned  to  fly  either  all  manual  approaches  or  all  autopilot-coupled  approaches,  but 
not  both.  Altitude  and  direction  were  the  within-subject  factors,  meaning  that  for  a  given 
analysis  each  subject  experienced  all  levels  for  that  factor. 

The  use  of  analysis  of  variance  requires  the  assumption  that  the  observations  from  all  test 
cells  are  normally  distributed  and  have  the  same  variance.  Similar  assumptions  about  the 
differences  between  observations  are  additionally  required  for  mixed-factor  analyses.  Before 
each  analysis,  the  assumptions  were  tested  using  the  appropriate  tests  provided  with  SPSS^m.  If, 
for  a  given  analysis,  the  observations  did  not  satisfy  the  required  assumptions,  then  the  data  were 
transformed.  Any  data  transformations  are  noted  in  the  results  section. 

Since  the  Phase  2  data  included  within-subject  replicates,  the  analysis  of  variance 
required  a  complete  set  of  data.  Missing  data  values  were  replaced  with  the  mean  value  for  that 
test  cell.  The  degree  of  freedom  for  the  denominator  was  then  reduced  by  the  number  of  missing 
values.  Any  missing  values  are  noted  in  the  results  section. 
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4.  DATA  ANALYSIS  RESULTS 


4.1  PILOT  PARTICIPATION 

Ten  crews  participate  in  each  phase  of  the  study.  Twenty-eight  pilots  participated,  with 
12  acting  as  subjects  in  both  phases.  The  subject  pilots  were  from  Northwest  Airlines,  United 
Airlines,  and  America  West  Airlines.  Pilot  participation  by  phase  is  listed  in  Table  4-1. 

Table  4-1.  Subject  Participation  by  Phase  of  Study 


Study 

Subject  Number  I 

Phase 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

1 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

2 

X 

X 

X 

X 

X 

X 

X 

X 

Study 

Subject  Number  I 

Phase 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

1 

X 

X 

X 

X 

X 

X 

2 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

-jJ 

4.2  LENGTH  OF  CONTROLLER  BREAKOUT  INSTRUCTIONS 

Two  controllers  participated  in  Phase  1  and  a  third  controller  participated  in  Phase  2. 
Table  4-2  lists  the  group  statistics  for  the  length  of  the  controller  transmissions  for  each  phase  of 
the  study.  The  breakout  instructions  during  Phase  2  included  additional  words  (<Aircraft>, 
TRAFFIC  ALERT),  and  averaged  one  second  longer  than  in  Phase  1. 

Table  4-2.  Duration  of  ATC  Breakout  Instruction  (seconds) 


Phase 

Count 

Median 

Mean 

S.D. 

Minimum 

Maximum 

1 

160 

4.1 

4.4 

0.9 

3.3 

7.6 

2 

177 

5.1  ^ 

5.0  ^ 

0.4 

CO 

6.8 

4.3  PRACTICE  EFFECTS 

Although  the  Phase  1  experimental  design  did  not  assign  within-subject  replicates, 
company  policies  for  the  participating  airlines  dictated  which  pilot  would  fly  Category  II 
approaches:  first  officer  for  two  airlines,  and  captain  for  the  third  airline.  Because  of  this 
requirement,  the  replicate  samples  for  the  autopilot-coupled  approaches  were  both  flown  by  the 
same  crewmember  and  could  be  used  as  within-subject  replicates  for  measunng  practice  effects. 
There  were  also  enough  within-subject  replicates  for  hand-flown  approaches  to  include  them  in 

the  analysis. 

Paired-sample  t  tests  for  the  mean  were  conducted  separately  for  the  Phase  1  hand-flown 
and  autopilot-coupled  data  in  case  there  was  a  difference  in  performance  over  time  for  one 
approach  mode,  but  not  for  the  other.  Six  performance  variables  were  tested:  dt_engine. 
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dt_inin_altitude,  dt_pitch,  dt_roll,  dt_toga,  and  dt_vertical_speed.  The  null  hypothesis  was  that 
there  was  no  change  in  performance  between  the  first  and  second  replicate.  If  the  calculated 
t  value  was  larger  than  the  critical  value  at  the  .05  level,  then  the  null  hypothesis  was  rejected, 
and  the  mean  difference  between  the  replicates  was  assumed  to  be  significantly  different  from 
zero.  No  t  values  were  significant  at  the  .05  level,  indicating  that  the  differences  between  the 
first  and  second  replicates  in  Phase  1  were  due  to  random  variability. 

The  Phase  2  experimental  design  included  within-subject  replicates  for  autopilot-coupled 
scenarios  at  500  feet  AGL  and  at  1800  feet  AGL.  The  same  variables  as  in  Phase  1,  except  for 
dt_toga,  were  tested  for  each  breakout  altitude  using  the  paired-sample  t  test.  Out  of  ten  t  tests, 
two  indicated  a  significant  difference  between  the  first  and  second  replicate.  The  t  test  indicated 
that  mean  time  to  increasing  engine  speed  at  500  feet  was  1.8  seconds  less  for  the  second  trial 
(t  =  -2.12,  d.f.  =  19,  p  =  .047;  Meani  =  15.2  seconds,  Meai^  =  13.4  seconds).  In  contrast,  the 
t  test  indicated  that  mean  time  to  minimum  altitude  at  1800  feet  was  6.8  seconds  longer  for  the 
second  trial  (t  =  -2.62,  d.f.  =  13,  p  =  .02;  Mean,  =  17.5  seconds,  Mean2  =  24.3  seconds). 

The  Phase  2  repeated-measure  data  were  also  grouped  by  whether  or  not  the  pilot 
participated  in  Phase  1  to  test  for  differences  attributable  to  previous  participation.  One  set  of 
paired  data  included  pilots  who  participated  in  Phase  1  and  the  other  set  included  pilots  who 
were  new  to  the  study.  These  t  tests  indicated  no  significant  differences  between  the  first  and 
second  replicate  for  either  level  of  experience. 

In  order  to  search  for  possible  practice  effects  not  identified  by  the  paired-sample  t-tests, 
all  data  were  qualitatively  reviewed.  First,  the  performance  variables  were  grouped  by  crew 
within  each  phase,  then  ordered  by  scenario  sequence.  The  ordered  variables  were  then  reviewed 
for  any  increasing  or  decreasing  trends  that  might  not  have  been  detected  by  the  t-tests.  The 
ordered  performance  variables  are  graphically  depicted  in  Appendix  F.  No  trends  were 
identified  for  either  phase.  If  a  straight  line  were  fitted  to  the  data  for  each  performance  variable, 
then  there  would  be  no  significant  increase  or  decrease  in  the  performance  variables  from  the 
beginning  to  the  end  of  each  session.  There  were  fluctuations  over  the  course  of  each  session, 
but  these  were  attributed  mainly  to  scenario  differences  and  to  variability  in  subject  responses. 

The  analysts’  notes  for  the  videotapes  were  also  reviewed.  Some  Phase  1  crews 
discussed  general  procedures,  but  there  were  no  specific  conversations  regarding  how  to  execute 
the  breakout.  No  pilot  comments  regarding  breakout  procedures  were  noted  from  the  Phase  2 
video  data.  These  observations,  together  with  the  analytical  results,  suggest  that  there  were  no 
observable  practice  effects  in  either  phase  of  the  study.  In  general,  response  times  did  not 
decrease  or  increase  over  time.  The  pilots  may  have  become  more  comfortable  with  the  breakout 
procedure  during  the  course  of  the  test  session,  but  the  learning  experience  did  not  significantly 
affect  the  observed  breakout  performance. 

4.4  GENERAL  OBSERVATIONS 

A  breakout  procedure  is  a  complex  set  of  responses  to  the  ATC  instruction.  There  are 
many  ways  for  the  crew  to  change  altitude  and  direction.  Similarly,  the  sequence  of  automation 
changes  depends  on  the  aircraft  configuration.  This  section  presents  the  trends  that  were 
observed  in  how  the  subjects  chose  to  execute  the  breakouts  and  in  the  relative  timing  of  the 
measured  aircraft  responses.  Two  pilot  inputs  were  recorded:  use  of  TOGA  and  use  of  autopilot. 
Other  pilot  inputs  could  not  be  measured.  The  analysts  observed  coarse  pilot  movements  such  as 
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reaching  for  the  heading  or  altitude  knob  on  the  video  recordings,  but  the  times  at  which  the 
pilots  entered  information  into  the  flight  director  could  not  be  determined. 

4.4.1  Use  of  Autopilot  and  TOGA 

In  both  study  phases,  use  of  the  autopilot  during  the  ATC-directed  breakout  was  at  the 
discretion  of  the  pilot  flying.  Following  hand-flown  approaches,  the  subjects  always  left  the 
autopilot  off  during  the  breakout.  Following  autopilot-coupled  approaches,  the  autopilot  was 
disconnected  before  the  start  of  turn  in  14  out  of  75  trials  in  Phase  1  (19  percent),  and  in  seven 
out  of  101  trials  in  Phase  2  (7  percent).  In  Phase  1,  ten  AP/HF  breakouts  were  at  decision 
altitude  or  500  feet  AGL.  Although  seven  of  the  20  pilots  in  Phase  1  disconnected  the  autopilot 
at  least  once,  nine  of  these  events  occurred  with  two  pilots.  In  Phase  2,  six  of  the  seven  events 
occurred  with  one  crew.  These  results  suggest  that  a  few  pilots  were  responsible  for  most  of  the 
AP/HF  breakouts  and  that  the  majority  of  the  subjects  preferred  to  leave  the  autopilot  connected 
during  the  breakout.  Because  of  the  small  number  of  subjects  conducting  AP/HF  breakouts, 
breakout  mode  was  not  considered  during  the  detailed  analysis. 

In  Phase  1,  the  subject  pilots  did  not  receive  any  training,  and  use  of  TOGA  thrust  was  at 
their  discretion.  In  Phase  2,  the  training  material  discouraged  the  use  of  TOGA  thrust  for 
climbing  breakouts  above  400  feet  AGL.  In  both  phases,  all  subjects  used  TOGA  thrust  during 
breakouts  at  decision  altitude,  and  did  not  use  TOGA  thrust  during  descending  breakouts. 

In  Phase  1,  TOGA  thrust  was  not  used  during  9  out  of  77  climbing  breakouts  above 
400  feet  AGL  (12  percent).  Four  of  these  events  were  for  one  subject.  Thus,  pilot  preference  in 
Phase  1  was  to  use  TOGA,  and  the  number  of  trials  in  which  it  was  not  used  was  too  small  to 
allow  comparison. 

In  Phase  2,  when  use  of  TOGA  was  discouraged,  TOGA  thrust  was  used  in  31  out  of  127 
climbing  breakouts  above  400  feet  AGL  (24  percent).  Of  these  events,  22  were  for  two  crews 
from  one  airline.  Because  use  of  TOGA  was  mainly  limited  to  four  subjects  in  Phase  2,  there 
was  not  enough  between-subject  variability  to  allow  comparisons. 

4.4.2  Climb  Performance 

Five  performance  variables  measured  A320  climb  response:  time  to  increasing  engine 
rpm  (dt_engine);  time  to  pitch  change  (dt.pitch);  time  to  vertical  speed  change 
(dt_vertical_speed);  time  to  minimum  altitude  (dt_min_altitude);  and,  distance  to  minimum 
altitude  (dz_min_altitude).  In  general,  the  subjects  continued  at  the  same  speed  and  vertical  rate 
during  descending  breakouts,  so  these  performance  variables  were  not  c^culated  for  descendmg 
scenarios.  Group  statistics  for  the  five  variables  are  presented  in  Appendix  G. 

Exploratory  analysis  indicated  two  trends.  First,  when  TOGA  thrust  was  used,  the  order 
of  events  was  generally  dt.engine,  then  dt_pitch,  and  then  dt_vertical_speed.  The  mean  time 
between  dt.engine  and  dt_pitch  was  0.6  second  for  manual  approaches  and  1.1  seconds  for 
autopilot-coupled  approaches.  The  mean  time  between  dt_pitch  and  dt_vertical_speed  was 
0.7  second  for  both  approach  modes.  For  those  trials  that  did  not  follow  this  trend,  TOGA  thrust 
was  applied  after  the  climb  had  been  initiated. 
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Second,  during  open  climb  following  autopilot-coupled  approaches,  the  order  of  events 
was  generally  dt_pitch,  then  dt_vertical_speed,  then  dt_engine,  with  mean  times  between  events 
of  2.1  seconds  and  2.6  seconds,  respectively.  Those  trials  that  did  not  follow  the  trend  were  at 
1800  feet  AGL,  and  the  turn  started  before  the  cliinb.  The  sequence  of  events  was  random  for 
open  climb  following  manual  approaches,  meaning  there  were  no  trends  observed  in  the  relative 
timings  of  dt_engine,  dt_pitch,  and  dt_vertical_speed  for  these  cases. 

Since  the  use  of  TOGA  thrust  affected  the  sequence  of  aircraft  responses,  it  was  possible 
that  it  also  affected  the  rate  of  climb.  In  order  to  check  for  this  possible  effect,  an  additional 
variable,  dt_transition,  was  calculated  during  the  analysis.  This  variable  was  calculated  as  the 
time  difference  between  start  of  vertical  acceleration  and  the  time  at  which  the  aircraft  vertical 
speed  was  zero;  dt_min_altitude  -  dt_vertical_speed.  Dt_transition  provided  an  estimate  of  the 
time  required  to  transition  from  a  negative  to  a  positive  vertical  speed,  or  from  a  descent  to  a 
climb. 

Figure  4-1  presents  the  dt_transition  data  grouped  by  approach  mode  and  climb  mode  for 
climbing  breakouts  above  400  feet  AGL.  The  data  from  Phase  1  and  Phase  2  are  shown 
separately  because  both  climb  modes  were  used  in  both  phases.  Data  from  decision  altitude  are 
not  included  because  TOGA  was  always  used  in  both  phases.  The  results  show  the  similarity  in 
distributions  between  the  two  phases,  although  the  numbers  of  samples  in  each  group  are 
different  for  each  phase.  In  the  majority  of  trials,  the  aircraft  transitioned  from  descend  to  climb 
in  less  than  5  seconds.  In  those  trials  in  which  TOGA  thrust  was  engaged  and  dt_transition  was 
longer  than  5  seconds,  TOGA  was  engaged  after  the  start  of  climb.  The  open-climb  trials  in 
which  dt_transition  was  greater  than  12  seconds  were  all  1800'  scenarios  in  which  pitch,  engine, 
and  vertical-speed  accelerations  were  slow.  In  each  of  these  cases,  vertical  speed  went 
below  1000  feet  per  minute  before  starting  to  increase.  Another  characteristic  was  that  engine 
speed  was  around  1600  rpm  at  the  start  of  the  breakout,  as  opposed  to  greater  than  3000  rpm  for 
other  500'  and  1800'  trials. 


HF  APPROACHES 


AP  APPROACHES 


Figure  4-1.  Transition  time  for  climbing  breakouts  above  400 feet  AGL. 
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4.4.3 


Turn  Performance 


Three  performance  variables  measured  A320  turn  response:  time  to  start  of  roll  (dt_roll); 
distance  to  start  of  roll  (dx_turn);  and,  maximum  roll  angle  magnitude  (maximum_roll). 
Distance  to  start  of  roll  was  proportional  to  time  to  start  of  roll  and  true  air  speed,  so  it  was  not 
analyzed  in  detail.  Statistics  for  the  three  variables  are  presented  in  Appendix  G. 

Maximum  roll  magnitude  was  consistently  between  23.7  and  25.6  degrees  for  breakouts 
in  which  the  autopilot  remained  engaged,  except  during  engine-out  scenarios.  Maximum  roll 
magnitude  was  more  variable  and  generally  smaller  for  the  engine-out  trials.  There  was  also 
greater  variability  for  hand-flown  breakouts,  although  the  mean  value  was  similar  to  the  mean 
for  autopilot-coupled  breakouts. 

For  hand-flown  breakouts,  maximum  roll  magnitude  exceeded  30  degrees  in  29  trials. 
These  trials  were  individually  reviewed  to  determine  if  there  were  any  common  traits.  Except 
for  one  pilot,  individual  performance  was  not  a  factor:  fourteen  of  the  28  subjects  exceeded 
30  degrees  at  least  once,  and  subject  8  exceeded  that  value  nine  times.  Fifteen  events  were  in 
Phase  1,  fourteen  were  in  Phase  2.  There  was  also  equal  distribution  between  HF/HF  and  AP/HF 
breakouts  and  between  first  and  second  half  of  the  session.  Most  of  the  large  roll  angles 
occurred  during  climbing  breakouts  above  400  feet  AGL  (24  trials).  Figure  4-2  presents  the  roll 
angle  data  for  the  nine  trials  in  which  maximum  roll  magnitude  was  greater  than  35  degrees.  In 
each  case,  roll  rate  was  constant  until  maximum  roll  angle  was  achieved.  Then  the  aircraft  rolled 
back  to  a  smaller  angle.  The  roll  angle  magnitudes  were  greater  than  30  degrees  for  five  to  seven 
seconds. 
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Figure  4-2.  Maximum  roll  magnitudes  greater  than  35  degrees.  The  negative  roll  angle 
values  indicate  a  left  roll.  Roll  histories  with  dotted  lines  are  for  subject  8. 


As  mentioned  in  Section  3.5.1,  two  algorithms  were  used  to  measure  time  to  start  of  roll. 
For  most  trials,  these  algorithms  returned  the  same  value  for  dt_roll.  In  24  cases,  the  algorithms 
returned  different  values.  These  trials  were  individually  reviewed  to  determine  if  there  were  any 
common  traits.  Six  events  were  the  right  engine-out  scenario;  and  the  aircraft  rolled  right  when 
thrust  was  applied.  Four  events  were  manual  approaches  with  climbing  breakouts.  It  was 
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unclear  whether  the  pilots  had  started  the  roll  then  rolled  back  slightly  or  if  the  rollback  was  an 
artifact  of  manual  control  using  the  side  stick. 

The  remaining  rollbacks  (14  events)  involved  autopilot  breakouts  at  500  and  1800  feet 
AGL.  These  were  distributed  among  nine  pilots.-  There  was  also  equal  distribution  between 
phases  of  study.  Eleven  events  occurred  in  the  second  half  of  the  session.  In  all  14  trials,  the 
new  heading  was  entered  into  the  flight  director  before  the  thrust  levers  were  set  to  TOGA. 
Once  TOGA  was  engaged,  the  flight  director  reverted  to  the  pre-programmed  go-around  track. 
The  crew  then  had  to  re-enter  the  new  heading  information.  In  these  trials,  the  start  of  turn  was 
delayed  4  to  18  seconds.  Figure  4-3  presents  the  roll  and  heading  data  for  two  of  these  trials  as 
illustrations  of  the  possible  delay  in  turn  away  from  the  approach  course  because  of  TOGA- 
induced  rollback. 


Figure  4-3.  Two  TOGA-induced  rollbacks.  Initial  heading  is  -2.5  degrees 

because  of  crosswinds. 


4.5  PHASE  1  ANALYSIS 

For  the  statistical  analysis,  subject  assignments  in  Phase  1  were  assumed  to  be  random 
with  respect  to  breakout  group  and  approach  mode.^  Replicates  for  the  decision  altitude  (DA), 
500  feet  AGL  (500'),  and  1800  feet  AGL  (1800')  breakout  groups  were  used  to  test  the  effect  of 


^  Assignments  were  not  completely  random  due  to  compliance  with  company  policy  and  to  ad  hoc  crew 
assignment  during  each  session. 
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altitude  on  turn  and  climb  performance.  Replicates  for  the  DA  and  engine-out  breakout  groups 
were  used  to  test  the  effect  of  cockpit  distractions  on  turn  and  climb  performance.  Replicates 
from  the  1800'  and  Descend  breakout  groups  were  used  to  test  the  effect  of  breakout  direction  on 
turn  performance  only.  The  effect  of  direction  on  climb  variables  was  not  tested  because  the 
pilots  did  not  change  vertical  speed  during  the  descending  breakouts. 

Five  dependent  variables  were  evaluated:  time  to  increasing  engine  rpm  (dt_engine);  time 
to  start  of  climb  (dt_vertical_speed);  time  to  TOGA  engage  (dt_toga);  altitude  loss 
(dz_min_altitude);  and,  time  to  start  of  roll  (dt_roll).  The  first  four  dependent  variables  were 
associated  with  climb  performance,  and  dt_roll  measured  turn  performance. 

4.5.1  Altitude  by  Approach  Mode 

Exploratory  analysis  indicated  that  the  variances  for  the  four  dependent  variables 
associated  with  climb  performance  were  much  smaller  at  decision  altitude  than  at  500'  or  1800'. 
These  variables  required  transformations  in  order  to  satisfy  the  ANOVA  assumption  of  equality 
of  variances.  The  best  transformations  were  the  inverses:  l/dt_engine;  l/dt_vertical_speed, 
l/dt_toga;  and,  l/dz_min_altitude. 

The  analysis  of  variance  indicated  a  significant  main  effect  for  altitude  with  all  five 
dependent  variables.  As  indicated  by  the  results  shown  in  Table  4-3,  the  four  climb  variables 
were  smallest  at  decision  altitude,  and  increased  significantly  with  each  altitude  group.  Turn 
performance  exhibited  the  opposite  trend:  mean  dt_roll  was  largest  at  decision  altitude  and 
smallest  at  1800  feet  AGL.  The  Tukey's  b  test  indicated  that  mean  time  to  start  of  turn  at 
decision  altitude  was  significantly  larger  than  the  1800'  mean,  but  that  it  was  not  significantly 
different  from  the  500'  mean.  The  500'  mean  also  was  not  significantly  different  from  the  1800' 
mean. 


Table  4-3.  Altitude  Main  Effect  (Altitude  by  Approach-Mode  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

F  VALUE ’ 

DA 

500’ 

1800' 

DIFFERENCES 

dt_engine  ^  (s) 

13.07 

3.3 

5.1 

7.1 

DA  <500' <1800’ 

dt_verticaLspeed  ^  (s) 

16.64 

4.6 

6.2 

8.2 

DA  <500’ <1800’ 

dt_toga  ^  (s) 

14.33 

CO 

4.8 

7.9 

DA  <500’ <1800’ 

dz_min_altitude  ^  (ft) 

34.72 

67 

89 

152 

DA  <  500’  <  1800’ 

dt_roll  (s) 

5.44 

11.3 

8.7 

6.9 

DA  >1800’ 

1  F.o5(2,69)  =  3.14 

2  Variable  was  transformed  for  the  analysis 


The  analysis  of  variance  also  indicated  a  significant  main  effect  for  approach  mode  for 
time  to  start  of  climb  and  time  to  start  of  roll.  As  listed  in  Table  4-4,  the  mean  values  for  hand- 
flown  approaches  were  significantly  smaller  than  the  mean  values  for  autopilot-coupled 
approaches.  The  calculated  F  valued  for  the  other  dependent  variables  were  smaller  than  the 
critical  value,  F.os,  indicating  that  the  differences  in  means  for  these  variables  were  random  and 
not  significant.  There  was  no  significant  interaction  effect  between  altitude  and  approach  mode. 
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Table  4-4.  Approach  Mode  Main  Effect  (Altitude  by  Approach  Mode  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

F  VALUE ’ 

HAND-FLOWN 

AUTOPILOT 

DIFFERENCES 

dt_engine  ^  (s) 

0.07 

4.8 

5.6 

None 

dt_verticaLspeed  ®  (s) 

5.74 

5.8 

6.9 

HF<AP 

dtjoga  ^  (s) 

1.19 

5.1 

5.4 

None 

dz_min_altitude  ^  (ft) 

0.16 

103 

101 

None 

dt_roll  (s) 

11.65 

7.3 

10.9 

HF<AP 

1  Fo5(1,69)  =  3.99 

2  Variable  was  transformed  for  the  analysis 


4.5.2  Direction  by  Approach  Mode 

Because  there  were  no  changes  to  the  dependent  variables  associated  with  climb 
performance  during  descending  breakouts,  only  turn  performance  was  assessed.  Exploratory 
analysis  indicated  unequal  variances  for  dt_roll  for  this  data  subset.  The  statistics  listed  in 
Appendix  G  show  that  the  mean  values  for  the  four  different  cases  were  all  larger  than  the 
median  values,  indicating  that  the  distributions  were  skewed  by  a  few  long  response  times.  The 
inverse  of  the  dependent  variable,  l/dt_roll,  was  used  in  the  analysis  to  compensate  for  the 
skewed  distributions.  The  analysis  of  variance  indicated  no  main  effect  for  direction,  meaning 
there  was  no  significant  difference  between  mean  values  for  climbing  and  descending  breakouts 
at  1800  feet  AGL  (F(l,39)  =  0.13;  p  >  0.05).  A  main  effect  was  indicated  for  approach  mode 
(F(l,39)  =  8.54;  p  <  0.05;  Meann?  =  7.2  s;  MeanAP  =  7.6  s).  The  mean  value  for  hand-flown 
approaches  was  smaller  than  for  autopilot-coupled  approaches.  This  result  was  the  same  as  the 
result  from  the  altitude  by  approach  mode  analysis.  There  was  no  interaction  effect  between 
direction  and  approach  mode. 

4.5.3  Distraction  by  Approach  Mode 

The  only  cockpit  indications  of  an  engine  failure  were  flashing  warning  lights  on  the 
Flight  Control  Unit  and  a  message  on  one  of  the  displays.  There  was  no  audible  alarm.  One 
crew  commented  afterwards  that  this  "isn't  much  indication  of  engine  failure."  The  failure  was 
not  verbally  acknowledged  in  six  trials.  In  six  other  trials,  the  crew  asked  to  continue  at  present 
heading  because  of  the  failure.  This  response  was  not  anticipated  during  the  experiment  design 
phase.  Although  operationally  significant,  the  straight-ahead  responses  did  not  provide  data  on 
climbing-tum  breakout  responses.  Thus,  these  straight-ahead  trials  were  not  used  in  the  analysis 
of  variance. 

Exploratory  analysis  indicated  that  all  dependent  variables  satisfied  the  assumption  of 
equal  variances,  so  no  transformations  were  required.  The  analysis  of  variance  indicated  a  mmn 
effect  for  distraction  level  with  dt_engine  and  dt_vertical_speed,  with  these  responses  occurring 
sooner  in  the  engine-out  scenarios.  The  analysis  information  is  listed  in  Table  4-5.  Although 
mean  dt_roll  was  1.3  seconds  less  for  the  DA  scenarios,  between-subject  variability  was  large 
enough  that  the  difference  was  not  statistically  significant. 
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Table  4-5.  Distraction  Main  Effect  (Distraction  by  Approach-Mode  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

F value’ 

DA 

ENG  OUT 

DIFFERENCES 

dt_engine  (s) 

21.25 

3.3 

2.3 

DA  >  ENG  OUT 

dt_verticaLspeed  (s) 

5.26 

4.6 

4.0 

DA  >  ENG  OUT 

dtjoga  (s) 

0.90 

3.4 

3.2 

None 

dz_min_altitude  (ft) 

1.99 

67 

62 

None 

dt_roll  (s) 

0.41 

11.3 

12.6 

None 

1  Fo5{1,36)  =  4.12 


The  analysis  of  variance  also  indicated  a  significant  main  effect  for  approach  mode  with 
three  dependent  variables;  dt_engine,  dt_toga,  and  dz_min_altitude.  These  dependent  variables 
were  smaller  for  autopilot-coupled  approaches.  The  results  are  listed  in  Table  4-6.  Although 
mean  dt_roll  was  3.5  seconds  less  for  hand-flown  approaches  between-subject  variability  was 
such  that  the  difference  was  not  statistically  significant.  There  was  no  interaction  effect  between 
distraction  level  and  approach  mode. 

Table  4-6.  Approach  Mode  Main  Effect  (Distraction  by  Approach-Mode  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

F  VALUE ’ 

HAND-FLOWN 

AUTOPILOT 

DIFFERENCES 

dt_engine  (s) 

21.92 

3.4 

2.4 

HF>AP 

dt_verticaLspeed  (s) 

0.36 

4.3 

4.4 

None 

dtjoga  (s) 

14.70 

3.8 

2.7 

HF>AP 

dz_min_altitude  (ft) 

6.59 

69 

60 

HF>AP 

dt_roll  (s) 

2.15 

10.2 

13.7 

None 

1  F.o5(1,36)  =  4.12 


4.6  PHASE  2  ANALYSIS 

Phase  2  was  designed  for  within-and-between  analysis  of  variance,  with  approach  mode 
as  the  between-subject  factor  and  breakout  group  as  the  within-subject  factor.  The  within- subject 
assignments  were  such  that  the  climbing-breakout  data  were  analyzed  in  two  sets.  The  first  set 
included  all  within-subject  replicates  at  decision  altitude  (DA),  500  feet  AGL  (500’),  and 
1800  feet  AGL  (1800’).  The  second  set  included  all  within-subject  replicates  at  500’,  1000  feet 
AGL  (1000’),  and  1800’.  Within-subject  replicates  for  the  1800’  and  Descend  breakout  groups 
were  used  to  test  the  effect  of  breakout  direction  on  time  to  start  of  Uim.  The  effect  of  breakout 
direction  on  speed  and  climb  variables  was  not  tested  because  the  pilots  did  not  change  vertical 
speed  during  the  descending  breakouts. 

Four  dependent  variables  were  evaluated:  time  to  increasing  engine  rpm  (dt_engine); 
time  to  increasing  vertical  speed  (dt_vertical_speed);  altitude  loss  (dz_min_altitude);  and,  time  to 
start  of  roll  (dt_roll).  Exploratory  analysis  indicated  that  dt_roll  and  dz_min_altitude  did  not 
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satisfy  the  assumption  of  homogeneity  of  variances  and  required  transformation.  The  best 
transformations  were  the  inverses:  l/dt_roll  and  l/dz_min_altitude.  The  other  test  variables  were 
not  transformed. 

4.6.1  Altitude  by  Approach  Mode 

The  analysis  of  variance  using  approach  mode  as  the  between-subject  independent 
variable  and  the  DA,  500’,  and  1800’  breakout  groups  as  the  within-subject  independent  variable 
yielded  significant  main  effects  for  both  approach  mode  and  altitude,  but  no  significant 
interaction  between  the  two  independent  variables.  The  analysis  information  for  the  main  effect 
of  approach  mode  is  listed  in  Table  4-7.  The  calculated  F  value  was  greater  than  the  critical 
value  for  all  dependent  variables,  indicating  that  the  difference  between  mean  values  was 
significant  and  not  random.  The  conclusion  was  that  all  analyzed  breakout  events  averaged  3  to 
4  seconds  less  for  hand-flown  approaches  than  for  autopilot-coupled  approaches.  Mean  altitude 
loss  for  hand-flown  approaches  was  25  feet  less  than  for  autopilot-coupled  approaches. 


Table  4-7.  Approach  Mode  Main  Effect  (DA-500'-1800'  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

F  VALUE ’ 

HAND-FLOWN 

AUTOPILOT 

DIFFERENCES 

dt_engine  (s) 

7.15 

7.6 

10.5 

HF<AP 

dt_verticaLspeed  (s) 

32.77 

6.7 

10.7 

HF<AP 

dz_min_altitude  ^  (ft) 

5.48 

138 

163 

HF<AP 

dt_roll  ^  (s) 

11.42 

6.1 

10.3 

HF<AP 

1  Fo5(1,14)  =  4.60 

2  Variable  was  transformed  for  the  analysis 


Table  4-8  lists  the  analysis  of  variance  information  for  the  altitude  main  effect.  The 
calculated  F  values  for  all  dependent  variables  were  larger  than  the  critical  value,  indicating  that 
the  mean  for  at  least  one  level  of  altitude  was  significantly  different  from  the  other  means. 
Because  altitude  was  the  within-subject  factor,  paired-sample  t-tests  were  used  to  determine 
which  levels  had  significantly  different  sample  means.  The  t  tests  yielded  a  significant 
difference  between  decision  altitude  and  the  other  two  altitudes  for  dt_engine  and 
dt_vertical_speed.  Both  events  averaged  5  to  6  seconds  less  at  decision  altitude  than  at  500’  and 
1800’.  There  was  no  significant  difference  between  500'  and  1800’  for  these  two  dependent 
variables.  Mean  distances  to  minimum  altitude  were  significantly  different  for  all  three  altitudes: 
mean  altitude  loss  increased  from  83  feet  at  decision  altitude  to  217  feet  at  1800  feet  AGL.  The 
results  for  time  to  start  of  roll  were  less  conclusive.  Although  mean  dt_roll  was  larger  at 
decision  altitude  than  at  the  other  two  altitudes,  it  was  significantly  different  from  only  the 
500’  mean.  The  t  tests  indicated  that  the  1800’  results  were  not  significantly  different  from  either 
the  DA  or  the  500’  results. 
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Table  4-8.  Altitude  Main  Effect  (DA-500'-1800'  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

F value’ 

DA  - 

500' 

1800' 

DIFFERENCES 

dt_engine  (s) 

15.60 

5.0 

11.2 

10.9 

DA  <  (500’, 1800’) 

dt_verticaLspeed  (s) 

14.73 

5.8 

10.0 

11.0 

DA  <  (500’, 1800’) 

dz_min_altitude  ^  (ft) 

66.20 

83 

150 

217 

DA  <  500’  <  1800’ 

dt_roll  ^  (s) 

5.21 

9.9 

6.7 

8.1 

DA  >  500’ 

1  F,o5(2,28)  =  3.34 

2  Variable  was  transformed  for  the  analysis 


Because  some  trials  were  rejected  during  the  data  validation  process,  two  cases  were 
missing  for  the  500'- 1000'- 1800'  data  set.  These  missing  data  were  replaced  by  the  appropriate 
means,  and  the  denominator  degree  of  freedom  was  reduced  by  two. 

The  analysis  of  variance  using  approach  mode  together  with  the  500',  1000',  and  1800' 
breakout  groups  as  independent  variables  yielded  a  main  effect  for  approach  mode  with  dt_roll 
and  dt_vertical_speed.  "The  analysis  information  is  presented  in  Table  4-9.  Although  the  means 
for  dt_engine  and  dz_min_altitude  were  smaller  for  hand-flown  approaches,  the  variability  in 
responses  was  large  enough  that  the  null  hypotheses  of  no  significant  difference  between  means 
were  not  rejected. 

Table  4-9.  Approach  Mode  Main  Effect  (500'-1000'-1800'  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

F value’ 

HAND-FLOWN 

AUTOPILOT 

DIFFERENCES 

dt_engine  (s) 

2.71 

10.3 

13.4 

None 

dt_vertical_speed  (s) 

5.46 

10.5 

13.2 

HF<AP 

dz_min_altitude  ^  (ft) 

0.01 

204 

218 

None 

dt_roll  ®  (s) 

26.65  ' 

5.1 

7.2 

HF  <  AP 

1  Fo5(1,12)  =  4.75 

2  Variable  was  transformed  for  the  analysis 


The  analysis  of  variance  using  the  500',  1000',  and  1800'  breakout  groups  also  yielded  a 
significant  main  effect  for  altitude  with  dt.vertical.speed  and  dz_min_altitude.  The  analysis 
information  is  presented  in  Table  4-10.  The  paired-sample  t  tests  yielded  a  significant  difference 
between  1800'  and  the  other  two  altitudes.  There  was  no  significant  different  between  500'  and 
1000'.  This  is  partly  consistent  with  the  DA-500'-1800'  analysis,  which  indicated  a  significant 
difference  between  500'  and  1800'  for  dz_min_altitude  but  not  for  dt_vertical_speed.  Mean  time 
to  start  of  roll  (dt_roll)  and  time  to  engine  rpm  increase  (dt_engine)  were  similar  for  the  three 
altitude  levels.  These  results  are  consistent  with  the  DA-500'- 1800'  analysis,  which  indicated  no 
significant  difference  between  500'  and  1800'  for  these  dependent  variables. 
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Table  4-10.  Altitude  Main  Effect  (500'-1000'-1800'  ANOVA) 


DEPENDENT 

CALCULATED 

MEAN  VALUE 

SIGNIFICANT 

VARIABLE 

rvalue’ 

500'  - 

1000' 

1800' 

DIFFERENCES 

dt_engine(s) 

1.30 

12.4 

12.4 

10.7 

None 

dt_verticaLspeed  (s) 

5.52 

10.6 

11.3 

13.8 

(500’,  1000')  <  1800' 

dz_min_altitude  ^  (ft) 

27.52 

170 

182 

280 

(500’,  1000’)  <  1800' 

dt_roll  ^  (s) 

0.38 

6.2 

5.8 

6.3 

None 

1  F.o5(2,26)  =  3.37 

2  Variable  was  transformed  for  the  analysis 


4.6.2  Direction  by  Approach  Mode 

The  analysis  of  variance  using  direction  and  approach  mode  as  independent  variables  was 
performed  on  the  transformed  turn  variable,  l/dt_tum.  The  results  indicated  a  significant  main 
effect  for  approach  mode.  The  mean  for  hand-flown  approaches  was  2.7  seconds  smaller  than 
for  autopilot-coupled  approaches  (F(l,14)  =  21.77;  Meaunp  =  5.2  s;  MeauAp  =  7.9  s).  The  results 
indicated  no  significant  main  effect  for  direction.  The  mean  value  for  climbing  breakouts  was 
similar  to  the  mean  value  for  descending  breakouts  (F(l,14)  =  0.01;  Meanigoo'  =  6.8  s;  MeanoEsc 
=  6.4  s).  There  was  no  significant  interaction  between  approach  mode  and  direction. 

4.7  COMPARISON  BETWEEN  PHASES 

In  Phase  1,  TOGA  thrust  was  used  in  all  breakouts  at  decision  altitude  and  in  88  percent 
of  the  climbing  breakouts  above  400  feet  AGL.  In  Phase  2,  TOGA  thrust  was  also  used  in  all 
breakouts  at  decision  altitude,  but  open  climb  mode  was  employed  by  16  out  of  20  subject  pilots 
during  climbing  breakouts  above  400  feet  AGL.  Analysis  of  variance  was  used  to  test  whether 
or  not  there  were  significant  differences  in  turn  or  climb  performance  between  the  two  study 
phases.  Time  from  start  of  ATC  instruction  to  start  of  roll,  dt_roll,  was  used  as  the  turn 
performance  metric.  The  metrics  for  climb  performance  were  time  from  start  of  ATC  instruction 
to  start  of  pitch  increase  (dt_pitch),  time  from  pitch  increase  to  increasing  vertical  speed 
(dt_vertical_speed  -  dt_pitch),  and  time  from  increasing  vertical  speed  to  positive  rate  of  climb 
(dt_transition). 

Each  variable  was  tested  for  each  of  the  breakout  groups  common  to  Phase  1  and 
Phase  2:  DA,  500’,  1800’,  and  Descend  (turn  performance  only).  All  variables  satisfied  the 
analysis  of  variance  assumptions  for  each  breakout  group,  therefore  no  transformations  were 
required.  Hand-flown  and  autopilot-coupled  trials  were  combined  (i.e.,  not  tested  separately, 
based  on  the  results  of  Sections  4.5  and  4.6. 

The  analysis  of  variance  indicated  no  change  in  turn  performance  between  Phase  1  and 
Phase  2.  The  calculated  F  values  for  each  of  the  four  breakout  groups  was  smaller  than  the  F.os 
critical  value,  indicating  no  significant  difference  between  dt_roll  distributions.  The  group 
means  and  calculated  F  values  are  listed  in  Table  4-11. 
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Table  4-1 1 .  Effect  of  Phase  on  Dt_roll 


MEANDT_ROLL(s)  | 

PHASE 

DA 

500' 

1800' 

DESCEND 

1 

11.3 

8.7  ■ 

6.9 

5.7 

2 

10.1 

6.4 

7.1 

6.4 

Calculated  F  Value 

F(1,41)  =  0.46 

F(1,62)  =  2.72 

F(1,60)  =  0.02 

F(1,32)  =  0.77 

The  analysis  of  variance  indicated  no  significant  difference  in  dt_pitch  between  Phase  1 
and  Phase  2  at  decision  altitude  (DA),  but  did  indicate  a  significant  difference  between  the  two 
phases  for  the  500'  and  1800'  breakout  groups.  The  calculated  F  value  was  less  than  the  F.os 
critical  value  for  the  DA  group,  but  larger  than  the  critical  values  for  the  500'  and  1800'  groups. 
The  group  means  and  F  values  are  listed  in  Table  4-12. 

A  comparison  between  the  mean  values  in  Table  4-11  and  Table  4-12  suggests  that,  on 
average,  climb  started  before  the  turn  during  breakouts  at  decision  altitude.  The  mean  statistics 
also  suggest  that,  on  average,  climb  started  before  or  at  the  same  time  as  the  turn  in  Phase  1  for 
breakouts  above  400  feet  AGL,  but  that  the  turn  was  initiated  before  climb  for  breakouts  above 
400  feet  AGL  during  Phase  2. 


Table  4-12.  Effect  of  Phase  on  Dt_pitch 


MEAN  DT_PITCH  (s) 

PHASE 

DA 

500' 

1800' 

1 

4.5 

5.6 

6.9 

2  1 

5.3 

9.0 

9.4 

Calculated  F  Value 

F(1,41)  =  1.99 

F(1 ,62)  =  22.73* 

F(1,60)  =  9.85* 

*  value  larger  than  F  05 


The  analysis  of  variance  indicated  similar  differences  between  Phase  1  and  Phase  2  for 
the  time  between  pitch  increase  and  vertical  speed  increase.  The  calculated  F  value  was  less  than 
the  critical  value  for  breakouts  at  decision  altitude,  indicating  no  significant  difference.  The 
calculated  F  value  was  larger  than  the  critical  value  for  climbing  breakouts  above  400  feet  AGL, 
indicating  that  the  time  from  pitch  increase  to  increasing  vertical  speed  was  longer  in  Phase  2. 
The  group  means  and  F  values  are  listed  in  Table  4-13. 


Table  4-13.  Effect  of  Phase  on  (Dt_vertical_speed  -  Dt_pitch) 


MEAN  (DT_VERTICAL_SPEED  -  DT.PITCH)  (s)  \ 

PHASE 

DA 

500' 

1800’ 

1 

0.1 

0.6 

1.4 

2 

0.5 

1.5 

3.0 

Calculated  F  Value 

F(1,41)  =  0.97  ^ 

F(1,62)  =  4.76*  ^ 

F(1,60)  =  3.80* 

*  value  larger  than  F  05 


Similarly,  the  analysis  of  variance  indicated  no  significant  difference  in  dt_transition  for 
breakouts  at  decision  altitude,  but  did  indicate  a  significant  difference  for  climbing  breakouts 
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above  400  feet  AGL.  The  mean  dt_transition  statistic  was  significantly  larger  in  Phase  2  than  in 
Phase  1,  indicating  that,  on  average,  it  took  longer  to  transition  to  a  positive  climb  rate  in  Phase  2 
at  altitudes  above  400  feet  AGL.  The  group  means  and  F  values  are  listed  in  Table  4-14. 


Table  4-14.  Effect  of  Phase  on  Dtjransition 


MEAN  DT^TRANSITION  (s)  | 

PHASE 

DA 

500‘ 

1800' 

1 

1.7 

1.9 

4.2 

2 

1.6 

4.4 

9.7 

Calculated  F  Value 

F(1,41)  =  0.00 

F(1,62)  =  13.80* 

F(1 ,60)  =  11.64* 

*  value  larger  than  F  05 


4.8  COMPARISON  WITH  PRM  RESULTS 

The  mean  and  maximum  value  statistics  for  time  to  start  of  roll  (dt_roll)  from  each  phase 
of  the  study  were  compared  to  the  statistics  for  the  B727  and  DC  10  data  used  in  the  Precision 
Runway  Monitor  Program  risk  analysis.  The  data  for  each  phase  were  grouped  by  breakout 
scenarios:  climbing  breakouts  at  decision  altitude  (DA);  climbing  breakouts  above  400  feet  AGL 
(500',  1000',  and  1800');  and,  descending  breakouts  at  1800  feet  AGL.  The  distraction  scenarios 
from  Phase  1  were  not  included  in  this  analysis.  For  each  phase,  a  given  approach  mode  was 
classified  as  acceptable  if  all  response  times  were  less  than  or  equal  to  17  seconds  and  mean  time 
to  start  of  roll  was  less  than  or  equal  to  8  seconds  for  the  three  scenario  groups. 

The  distributions  of  times  from  start  of  ATC  transmission  to  start  of  roll  are  presented  in 
Figure  4-4  for  Phase  1.  The  statistics  for  each  approach  mode  are  also  presented.  Mean  time  to 
start  of  turn  was  acceptable  for  only  two  conditions:  hand-flown  approaches  followed  by 
climbing  breakouts  above  400  feet  AGL  and  autopilot-coupled  approaches  followed  by 
descending  breakouts.  Except  for  descending  AP  mode  breakouts,  the  mean  statistics  were  two 
or  more  seconds  larger  than  the  medians  (50*  percentile)  because  of  the  long  response  times.  In 
Phase  1,  16  trials  out  of  136  had  a  time  to  start  of  roll  greater  than  17  seconds.  Table  4-15  lists 
data  for  each  of  these  events.  Six  were  at  decision  altitude,  two  were  descending  breakouts,  ^d 
eight  were  climbing  breakouts  above  400  feet  AGL.  Only  two  of  the  ten  autopilot-coupled  trials 
listed  in  Table  4-15  had  TOGA-induced  rollbacks.  Seven  of  the  long  response  times  were  for 
subject  6,  while  the  other  nine  events  were  attributed  to  eight  pilots. 

The  distributions  and  statistics  for  time  to  start  of  roll  are  presented  in  Figure  4-5  for 
Phase  2.  All  mean  statistics  were  less  than  8  seconds  except  for  autopilot  mode  breakouts  at 
decision  altitude.  There  were  three  trials  with  time  to  start  of  roll  greater  than  17  seconds.  Two 
were  hand-flown  scenarios  (DA  and  1800')  with  subject  6  as  pilot  flying.  The  other  was  an 
autopilot-coupled  scenario  at  decision  altitude.  All  three  trials  were  in  the  middle  of  the  session. 
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Table  4-15.  Phase  1  Times  to  Start  of  Turn  Greater  Than  17  Seconds 


Approach 

Mode 

Breakout 

Group 

Sequence 

Number 

Pilot 

DT  ROLL 

(s) 

DT.TOGA 

(s) 

First  roil 
(S) 

HF 

DA 

4 

6 

18.7 

3.7 

— 

HF 

1 

6 

19.0 

4.7 

— 

HF 

17 

6 

21.0 

4.7 

4.0 

HF 

1800' 

5 

6 

23.7 

6.3 

— 

HF 

DESC 

2 

3 

27.0 

3.0 

HF 

DESC 

2 

9 

26.3 

2.7 

— 

AP 

DA 

9 

6 

19.0 

3.3 

-- 

AP 

DA 

3 

6 

3.3 

— 

AP 

DA 

13 

11 

27.0 

2.0 

— 

AP 

DA 

13 

14 

3.3 

— 

AP 

DA 

13 

18 

1.0 

— 

AP 

500’ 

12 

4 

21.0 

4.7 

— 

AP 

500‘ 

13 

6 

18.0 

4.0 

-  — 

AP 

500' 

8 

16 

18.3 

5.0 

— 

AP 

1800' 

15 

4 

5.3 

6.0 

AP 

1800' 

15 

19 

5.0 

4.9  SUMMARY 

Approach  mode  affected  turn  performance  in  both  study  phases,  with  mean  times  to  start 
of  turn  following  hand-flown  approaches  being  2  to  4  seconds  smaller  than  mean  times  following 
autopilot-coupled  approaches  at  all  altitudes.  Approach  mode  had  a  greater  effect  on  climb 
performance  in  Phase  2  than  in  Phase  1.  In  Phase  1,  mean  time  to  increasing  vertical  speed  was 
significantly  less  for  hand-flown  approaches;  but  there  were  no  significant  differences  between 
mean  values  for  the  other  three  climb  variables.  In  Phase  2,  however,  all  four  climb  variables 
were  significantly  smaller  for  hand-flown  approaches. 

Altitude  also  affected  performance  in  both  phases.  Mean  values  for  climb  variables  were 
least  at  decision  altitude,  and  increased  with  increasing  altitude.  The  opposite  result  was 
observed  for  time  to  start  of  turn.  In  both  phases,  mean  dt_roll  was  significantly  more  at 
decision  altitude  than  at  the  altitudes  above  400  feet  AGL.  There  was  no  significant  difference 
in  turn  performance  between  500  feet  AGL  and  1800  feet  AGL. 

Breakout  direction  did  not  affect  turn  performance.  In  both  study  phases,  mean  time  to 
start  of  turn  was  statistically  similar  for  descending  and  climbing  breakouts  at  1800  feet  AGL. 

The  engine-out  distraction  also  had  little  effect  on  climbing-turn  breakout  performance. 
The  analysis  of  variance  indicated  similar  means  for  time  to  start  of  turn  (dt_roll),  time  to  TOGA 
(dt_toga),  and  vertical  loss  (dz_min_altitude).  Mean  time  to  increasing  engine  rpm  (dt_engine) 
and  mean  time  to  start  of  climb  (dt_vertical_speed)  were  less  for  the  distraction  scenarios.  This 
result,  however,  may  be  an  artifact  of  the  crews  responding  to  the  engine  failure  rather  than  an 
indicator  of  performance  differences.  The  six  straight-ahead  responses  were  not  used  in  this 
analysis. 
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COUNT  COUNT  COUNT 


TIME  TO  START  OF  ROLL  (s) 


Climbing  breakouts  at  decision  altitude 


Approach 

Mode 

Count 

Mean 

(s) 

Medlar 

(s) 

S.D. 

(s) 

Max 

(s) 

HF 

20 

4.7 

18.7 

AP 

20 

14.2 

12.3 

6.6 

30.3 

Climbing  breakouts  above  400  feet  AGL 


Approach 

Mode 

Count 

Mean 

(s) 

Medlar 

(s) 

S.D. 

(s) 

Max 

(s) 

HF 

39 

6.6 

4.4 

5.0 

23.7 

AP 

38 

9.4 

7.0 

5.4 

23.3 

Descending  breakouts  at  1800  feet  AGL 


Approach 

Mode 

Count 

Mean 

(S) 

ill 

Bl 

HF 

9 

9.2 

4.3 

9.9 

27.0 

AP 

10 

6.7 

5.7 

3.0 

13.0 

Figure  4-4.  Times  to  start  of  roll  for  Phase  1  breakouts. 
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COUNT  COUNT  COUNT 


TIME  TO  START  OF  ROLL  (s) 


Climbing  breakouts  at  decision  altitude 


1^1 

8 

6.6 

18.1 

AP 

10 

12,8 

11.5 

5.4 

26.7 

Climbing  breakouts  above  400  feet  AGL 


Approach 

Mode 

Count 

Mean 

(S) 

Median 

(S) 

S.D. 

(s) 

Max 

Cs) 

HF 

45 

5.3 

4.5 

2.8 

21.1 

AP 

82 

7.6 

6.7 

2.5 

16.8 

Descending  breakouts  at  1800  feet  AGL 


Approach 

Mode 

Count 

Mean 

(S) 

Median 

(S) 

S.D. 

(s) 

Max 

(S) 

HF 

8 

5.3 

5.1 

0.9 

6.9 

AP 

9 

7.3 

6.9 

1.5 

10.4 

Figure  4-5.  Times  to  start  of  roll  for  Phase  2  breakouts. 
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The  analysis  of  variance  comparing  Phase  1  to  Phase  2  indicated  no  change  in  turn 
performance.  Mean  times  to  start  of  turn  were  statistically  similar  between  phases  for  all 
breakout  groups.  In  contrast,  the  results  indicated  a  significant  difference  between  phases  in 
climb  performance  for  altitudes  above  400  feet  AGL.  The  dt_pitch  mean  statistics  suggested  that 
the  pilots  waited  longer  to  initiate  the  climb  in  Phase  2.  In  addition,  the  results  indicated  that  the 
aircraft  response  generally  was  more  sluggish  in  Phase  2.  The  mean  time  used  between  the  start 
of  the  climb  response  and  the  time  at  which  positive  climb  rate  was  achieved  was  significantly 
longer  in  Phase  2  than  in  Phase  1.  A  comparison  between  the  mean  turn  and  climb  statistics  also 
suggests  that  in  Phase  1,  the  subjects  generally  started  the  climb  before  the  turn;  whereas  in 
Phase  2,  the  subjects  started  the  turn  before  the  climb.  There  were  no  significant  differences  in 
climb  performance  at  decision  altitude  between  Phase  1  and  Phase  2. 

Phase  1  turn  performance  did  not  satisfy  either  of  the  test  requirements.  Mean  times  to 
start  of  turn  were  generally  larger  than  8  seconds,  and  twelve  percent  of  the  trials  had  a  dt_roll 
value  of  greater  than  17  seconds. 

Turn  performance  was  better  in  Phase  2.  Except  for  autopilot-coupled  breakouts  at 
decision  altitude,  all  Phase  2  mean  times  to  start  of  roll  were  less  than  8  seconds.  The  number  of 
long  times  to  start  of  turn  was  also  smaller.  The  three  trials  with  dt_roll  greater  than  17  seconds 
had  similar  circumstances.  Two  of  the  three  occurred  at  decision  altitude,  and  two  of  the  three 
were  manual  breakouts  for  one  subject  pilot. 

Use  of  TOGA  thrust  had  an  apparent  effect  on  both  turn  and  climb  performance.  First,  if 
TOGA  thrust  was  applied  after  the  new  heading  was  selected  during  autopilot-coupled 
breakouts,  then  the  flight  director  would  revert  to  the  pre-programmed  go-around  track.  The 
crew  then  had  to  re-enter  the  new  heading.  Sometimes  this  event  resulted  in  a  significant  delay 
to  start  of  turn.  Second,  if  TOGA  thrust  was  not  used  and  the  aircraft  remained  in  open-climb 
mode,  then  the  transition  from  descend  to  climb  sometimes  took  longer  than  expected  at  1800 
feet  AGL.  This  event  appeared  to  be  related  to  the  initial  engine  speed  and  altitude.  Long 
transition  times  occurred  in  both  manual  and  autopilot-coupled  scenarios. 
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5.  SURVEY  RESULTS 


Following  participation  in  A320  Phase  1  and  A320  Phase  2,  subjects  were  asked  to 
complete  a  survey.  Each  phase  had  its  own  survey  that  included  questions  consistent  with  the 
intent  of  that  particular  phase  of  study. 

The  questions  in  the  Phase  1  Survey  focused  on  assessing  the  level  of  difficulty  of 
breakouts  and  identifying  differences  in  the  level  of  difficulty  that  may  be  attributable  to  where 
the  breakout  occurred,  i.e.,  at  decision  altitude,  inside  the  outer  marker,  and  outside  the  outer 
marker.  The  Phase  1  Survey  was  also  designed  to  elicit  responses  regarding  the  specific 
conditions  that  may  have  influenced  pilot  responses,  including;  altitude,  configuration,  air  speed, 
company  policy,  and  passenger  comfort. 

The  primary  focus  of  the  Phase  2  Survey  was  to  assess  the  potential  value  added  by  the 
training  and  materials  provided  to  increase  situational  awareness.  Questions  were  included 
regarding  the  general  level  of  difficulty  of  executing  breakouts  but  did  not  focus  on  variations  on 
the  level  of  difficulty  as  a  function  of  where  the  blunder  occurred  or  what  specific  conditions 
may  have  influenced  pilot  response. 

Section  5.1  reports  the  results  of  the  Phase  1  Survey  and  Section  5.2  reports  the  results 
of  the  Phase  2  Survey.  In  reporting  results,  verbatim  comments  made  by  the  pilot-subjects  are 
included.  In  some  cases,  pilots  used  abbreviations  for  commonly  used  terms.  The  translation  of 
an  abbreviation  is  provided  in  parenthesis  by  the  data  analyst  the  first  time  a  particular 
abbreviation  is  used. 

5.1  SURVEY  RESULTS  FOR  PHASE  1 

Ten  crews,  each  consisting  of  a  captain  and  first  officer,  participated  in  Phase  1.  Each 
participating  pilot  was  assigned  a  subject-number  to  assure  the  anonymity  of  his  or  her 
responses.  Pilots  were  asked  to  list  their  total  flight  time  and  the  total  hours  in  the  aircraft  type 
A320.  Table  5-la  and  Table  5-lb,  respectively,  list  the  total,  mean,  and  range  for  "total  flight 
time"  and  "hours  in  type"  for  both  captains  and  first  officers.  The  values  were  calculated  from 
the  data  provided  by  all  pilots  who  responded  to  the  questions.  When  the  number  of  respondents 
(n)  is  less  than  10,  this  indicates  that  either  the  pilot  did  not  respond  to  the  question  or  that  the 
response  was  illegible. 


Table  5-1  a.  Flight  Time  (hours) 


POSITION 

TOTAL  FLIGHT  TIME 

MEAN 

RANGE 

Captain 
(n  =  8) 

111,250 

12,361 

5,600  to  20,000 

First  Officer 
{n  =  5) 

78,000 

8,666 

2,500  to  14,000 
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Table  5-1  b.  Hours  in  Type  (A320) 


POSITION 

TOTAL  HOURS  IN  TYPE 

MEAN 

RANGE 

Captain 
(n  =  9) 

14,450 

1,605 

400  to  3,000 

First  Officer 
(n  =  5) 

13,385 

1,487 

85  to  3,000 

Each  crew  worked  together  in  completing  the  survey.  It  was  originally  intended  that  one 
survey  would  be  completed  by  each  pilot  independently.  However,  in  administering  the  survey 
site  staff  asked  each  crew  to  complete  the  survey  together.  Therefore,  although  twenty  pilots 
participated  in  the  study;  the  total  number  of  completed  surveys  is  ten. 

Survey  Items  1  through  4, 

Items  1  through  4  of  the  survey  refer  to  various  types  of  breakouts  at  various  points  along 
the  approach  path.  For  each  of  these  items  the  crews  were  asked  to  give  their  opinions  specific 
to  flying  a  coupled  approach  or  hand-flying  the  approach.  Items  1  through  3  refer  to  climbing 
breakouts,  while  Item  4  refers  to  a  descending  breakout.  Crews  were  asked  to  rate  the  level  of 
difficulty  of  each  of  the  cases  below  by  circling  a  number  on  a  5-point  scale: 


0 

1 

2 

3 

4 

Not  at  all 

Somewhat 

Moderately 

More  than 

Very 

Difficult 

Difficult 

Difficult 

Moderately 

Difficult 

Difficult 

Survey  Item  la.  Rate  the  difficulty  of  a  climbing  breakout  at  DH/DA  (Decision  Height/ 
Decision  Altitude)  during  the  simulation  when  flying  a  CQppkd 
approach. 

Survey  Item  lb.  Rate  the  difficulty  of  a  climbing  breakout  at  DH/DA  during  the 
simulation  when  hand  flying  (flight  director)  the  approach. 


Survey  Item  2a. 


Survey  Item  2b. 


Survey  Item  3a. 


Survey  Item  3b. 


Rate  the  difficulty  of  a  climbing  breakout  inside  the  outer  marker 
during  the  simulation  when  flying  a  coupled  approach. 

Rate  the  difficulty  of  a  climbing  breakout  inside  the  outer  marker 
during  the  simulation  when  hand  flying  (flight  director)  the  approach. 

Rate  the  difficulty  of  a  climbing  breakout  outside  the  outer  marker 
during  the  simulation  when  flying  a  coupled  approach. 

Rate  the  difficulty  of  a  climbing  breakout  outside  the  outer  marker 
during  the  simulation  when  hand  flying  (flight  director)  the  approach. 
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Survey  Item  4a. 


Rate  the  difficulty  of  a  descending  breakout  outside  the  outer  marker 
during  the  simulation  when  flying  a  coupled  approach. 

Survey  Item  4b.  Rate  the  difficulty  of  a  descending  breakout  outside  the  outer  marker 
during  the  simulation  when  hand  flying  (flight  director)  the  approach. 

Figure  5-1  graphically  illustrates  the  results.  These  data  are  reported  as  the  percentage  of 
crews  selecting  each  specific  rating  for  each  specific  item.  Of  note  is  the  fact  that  crews 
generally  rated  the  difficulty  o  the  breakout  as  "not  at  all"  or  "somewhat  difficult"  and  never 
rated  it  as  "more  than^moderately"  or  "very  difficult." 

Climbing  Breakouts  at  Climbing  Breakouts 

Decision  Altitude  Inside  the  Outer  Marker 


1  2  3  4  5 

Not  At  All  Somewhat  Moderately  Very 


Climbing  Breakouts 
Outside  the  Outer  Marker 


1  2  3  ® 

Not  At  All  Somewhat  Moderately  |"  Very 


Descending  Breakouts 
Outside  the  Outer  Marker 


■ 

Coupled 

0 

Hand  Flown 

0 

“T 

5 


2  3  4 

Not  At  All  Somewhat  Moderately  Very 


1  2  3  4  5 

Not  At  All  Somewhat  Moderately  Very 


Figure  5-1.  Crew  ratings  of  difficulty  of  breakouts. 
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Following  each  of  these  four  survey  items,  the  crews  were  asked  to  answer  the  following 
question: 

When  you  were  directed  by  ATC  to  turn  inunediately,  did  you  use  the  following  as  a  basis 
for  your  decision  to  start  turning? 

A  list  of  attributes  was  provided  and  crews  were  asked  to  respond  with  a  "yes"  or  "no"  to  each  of 
the  attributes,  including:  altitude,  aircraft  configuration,  air  speed,  company  policy,  and 
passenger  comfort.  Table  5-2  lists  the  percentage  of  "yes"  responses,  indicating  that  an  attribute 
was  part  of  the  basis  on  which  the  crew  decided  to  make  the  turn.  For  example,  as  seen  below, 
30%  of  the  crews  said  that  "yes,"  altitude  was  a  factor  in  making  the  decision  to  start  turning  in 
the  case  of  a  climbing  breakout  at  decision  altitude. 


Table  5-2.  Basis  for  Deciding  to  Turn 
(Percentage  of  "yes"  responses) 


TYPE  OF 
BREAKOUT 

ALTITUDE 

CONFIGURATION 

AIR 

SPEED 

COMPANY 

POLICY 

PASSENGER 

COMFORT 

OTHER 

Climbing  Breakout 
at  Decision  Altitude 

30 

20 

10 

20 

0 

30 

Climbing  Breakout 
inside  the  Outer 
Marker 

20 

20 

0 

20 

0 

20 

Climbing  Breakout 
outside  the  Outer 
Marker 

10 

10 

0 

10 

10 

20 

Descending 
Breakout  outside 
the  Outer  Marker 

20 

10 

0 

20 

10 

10 

The  number  of  "yes"  responses  does  not  vary  for  the  most  part,  except  for  some 
differences  reported  regarding  the  consideration  of  altitude  and  air  speed  on  climbing  breakouts 
at  decision  altitude.  However,  the  differences  are  small. 

To  elaborate  on  what  was  meant  by  "other"  in  response  to  Items  Ic,  2c,  3c,  and  4c,  one 
crew  that  responded  "yes"  made  comments  indicating  that  a  consideration  was  the  ATC 
instruction  to  turn  immediately  and  the  sense  of  urgency  in  the  voice  of  the  controller. 

As  part  of  Item  4c  one  additional  attribute  was  listed:  "Obstacles/Minimum  Vectoring 
Altitude."  Two  (20%)  of  the  ten  crews  said  "yes"  that  this  was  a  consideration.  A  comment 
accompanied  one  of  the  "yes"  responses: 

•  Questioned  to  ensure  terrain  clearance. 

Item  4c  also  contained  the  following  question:  Was  thrust  management  a  factor?  Six 
crews  (60%)  reported  "no,"  while  four  crews  (40%)  reported  "yes."  Three  of  the  crew  reporting 
"yes"  made  the  following  comments: 


•  Decision  not  to  use  TOGA(take  off/go  around  thrust)  if  not  required,  i.e.,  not 
low  altitude. 

•  Cannot  use  TOGA,  another  mental  step. 

•  No  TOGA  thrust. 

Survey  Item  5.  Does  your  company  direct  a  minimum  altitude  for  all  turns? 

Eight  of  the  crews  (80%)  responded  "yes,"  the  company  directs  a  minimum  altitude  for 
all  turns.  Responses  were  sorted  by  airline.  The  crews  from  Northwest  Airlines  reported  that  the 
company-directed  minimum  altitude  for  all  turns  is  500  feet,  unless  operationally  required.  One 
of  the  crews  from  America  West  responded  "no"  and  the  other  responded  "400  feet,  unless  an 
emergency  immediate  turn  is  needed."  Of  the  two  crews  from  United  Airlines,  one  responded 
"no"  and  the  other  responded  "400  feet." 

Survey  Item  6.  Given  the  runway  spacing  during  the  test,  if  you  knew  that  another 
aircraft  was  on  a  simultaneous  parallel  instrument  approach  in  IMC 
would  that  have  made  any  difference  in  your  response  to  an  ATC 
instruction  to  make  an  immediate  turn  at  low  altitude? 

Six  of  the  crews  (60%)  responded  "yes"  and  four  of  the  crews  (40%)  responded  "no." 
Only  one  crew  responding  "no"  commented:  "no,  we  depend  on  ATC  to  give  required  vector. 
Comments  following  a  "yes"  response  were: 

•  Depends  on  direction  of  turn  —  into  or  away  from  traffic. 

•  In  this  case,  ELS  (Instrument  Landing  System)  36L  (Left),  an  instruction  to 
turn  right  would  need  some  thought. 

•  If  turning  into  other  aircraft  —  I'd  sure  ask. 

•  We  may  have  clicked  A/P  (autopilot)  off  to  get  quicker  turn. 

•  Especially  toward  parallel  runway. 

Survey  Item  7.  Would  your  reaction  to  an  ATC  instruction  to  make  an  immediate  turn 
at  low  altitude  be  any  different  if  in  addition  to  the  circumstances 
described  in  Item  6  above,  you  also  had  a  written  procedure  which 
emphasized  the  need  for  an  immediate  response?  For  example:  "An 
immediate  pilot  response  is  expected  and  required.  Execution  of  these 
ATC  instructions  must  be  as  rapid  as  practical." 

Six  of  the  crews  (60%)  responded  "no"  and  four  of  the  crews  (40%)  responded  "yes." 
The  crews  who  responded  "no"  did  not  elaborate  on  their  responses.  However,  one  of  the  pilots 
who  responded  "no"  commented  that  even  though  he  said  "no,"  he  thought  that  written 
procedures  would  be  beneficial.  The  comments  following  a  "yes  response  were. 

•  Makes  you  more  aware  of  the  concerns  of  proximity  to  other  aircraft. 

•  We  need  some  direction  and  practice/training. 

•  Might  change  the  priorities. 
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Survey  Item  8.  What,  if  any,  A320  aircraft  iimitations  do  you  think  could  cause  an 
inherent  unwanted  delay? 

Two  of  the  crews  (20%)  had  no  comment.  Eight  of  the  crews  (80%)  made  the  following 
comments: 

•  A320  ECU  (Flight  Control  Unit)  should  be  modified  to  allow  selecting  a 
HDG  (heading)  without  TOGA  causing  it  to  reset  GA  TRK  (go-around 
track). 

•  TOGA  thrust  selects  go-around  NAV  (navigation)  track  —  requires  pilots  to 
select  heading  to  comply  with  ATC  headings  —  requires  good  auto  flight 
management  procedures. 

•  Too  much  reliance  on  the  auto  flight  system.  The  pilot  must  initiate  this 
action  without  depending  on  or  waiting  for  autoflight  response. 

•  When  you  pull  HDG  knob  to  activate  HDG  then  go  TOGA  on  throttles  then 
HDG  goes  to  RWY  (runway)  track  —  you  then  have  to  pull  HDG  again. 

•  Crews  must  be  extremely  aware  of  flight  mode,  especially  "GA  Track," 
changes.  Normal  law  may  limit  roll  rate. 

•  Autopilot  is  programmed  for  smoothness. 

•  When  advancing  thrust  to  GA,  this  wipes  out  all  previous  inputs  on  the  FCU 
and  may  stop  the  turn  to  go  to  GA  track. 

•  A320  aircraft  limitation  —  load  demand  limitation  of  autopilot  limits 
maneuvers.  During  GA  tried  going  TOGA  as  well  as  open  climb.  Each 
way  gives  different  initial  responses  which  might  affect  the  desired  action  of 
the  aircraft.  Might  want  to  consider  recommended  action  as  far  as  using 
TOGA  or  just  using  FCU  inputs  for  breakout  /  GA  execution. 

Survey  Item  9.  What  degree  of  urgency  does  the  term  "hnmediate"  convey  to  you? 

Comments  by  aU  the  crews  indicated  that  the  term  "immediate"  was  associated  with  a 
high  or  very  high  degree  of  urgency.  Those  comments  were: 

•  High  degree  of  urgency. 

•  Emergency  turn. 

•  Right  now,  consistent  with  maintaining  aircraft  control. 

•  Now! 

•  Urgent. 

•  High  degree. 

•  Very  high. 

•  Very  high  degree. 

•  Most  urgent. 

•  Immediate,  start  turn  and  climb  for  safety. 
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Survey  Item  10.  During  the  testing,  did  you  develop  any  strategy  for  making  a  decision 
to  turn  and  then  executing  the  turn? 

Eight  of  the  crews  (80%)  responded  "yes,"  while  two  of  the  crews  (20%)  responded  "no." 
The  crews  who  responded  "yes"  made  the  following^  comments: 

•  With  experience  you  begin  to  see  which  procedures  work  better  than  others. 

•  TOGA  thrust  first,  then  turn. 

•  Decided  to  fly  the  aircraft,  then  turn  when  conditions  were  safe. 

•  When  you  become  conditioned  to  the  fact  that  you  are  going  to  do  a  go- 
around  you  are  more  mentally  alert  for  this  condition. 

•  Get  turn  going  (left/iight),  then  confirm  exact  clearance. 

•  Heading  first  because  the  autopilot  is  already  set-up  for  climb. 

•  Responded  to  "immediately"  -  immediately. 

•  Start  the  climb  or  descent  and  direction  of  turn  and  then  "fine  tune"  after 
established. 

Survey  Item  11.  Do  you  think  that  training  or  better  situational  awareness  would  have 
enhanced  your  performance  during  the  ATC-directed  breakout? 

Nine  of  the  crews  (90%)  said  "yes"  and  one  crew  (10%)  said  "no."  Comments  regarding 
the  need  for  training  or  better  situational  awareness  were: 

•  Training  should  definitely  be  conducted  for  decreased  separation  and 
simultaneous  parallel  approach-type  operations,  should  these  operations 
become  widespread  and  common.  Without  training,  much  more 
disorganization  in  the  flight  deck  is  likely,  and  safe  completion  is  more 
problematic. 

•  Possible  instruction  concerning  procedures  for  flying  if  directed  to  do  a  non¬ 
standard  maneuver. 

•  This  is  not  the  typical  way  you  are  trained  to  do  GA. 

•  Introduce  breakouts  into  simulation  scenarios,  emphasizing  aircraft 
configuration  and  mode  changes,  especially  automated  cockpits. 

•  Training  always  helps. 

•  If  we  had  known  impact  with  another  aircraft  was  close  at  hand,  we  would 
have  taken  a/c  (aircraft)  off  autopilot  to  get  quicker  turn. 

•  Knowledge  of  simultaneous  ILS  in  progress,  closely  spaced  underscores  the 
need  for  immediate  response. 
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Survey  Item  12.  Any  other  comments? 

Two  crews  commented  that  the  simulation  was  a  good  training  experience  and  provided 
the  following  responses: 

•  This  was  a  good  exercise  and  proved  that  there  are  limits  to  the  immediate 
needs  of  a  go-around  in  the  autoflight  system. 

•  Great  training  -  got  us  thinking  about  other  than  training  strategies. 

5.2  SURVEY  RESULTS  FOR  PHASE  2 

Ten  crews,  each  consisting  of  a  captain  and  first  office,  participated  in  A320  Phase  2. 
The  pilots  of  eight  of  the  ten  crews  completed  the  Phase  2  Survey.  Each  participating  pilot  was 
assigned  a  subject-number  to  assure  the  anonymity  of  his  or  her  responses.  Pilots  were  asked  to 
list  their  total  flight  time  and  the  total  hours  in  the  aircraft  type  tested,  i.e.,  A320.  Table  5-3a  and 
Table  5-3b,  respectively,  list  the  total,  mean,  and  range  for  "total  flight  time"  and  "hours  in  type" 
for  both  captains  and  first  officers.  The  values  were  calculated  from  the  date  provided  by  all 
pilots  who  responded  to  the  questions.  When  the  number  of  respondents  (n)  is  less  th^  8,  this 
indicates  that  either  the  pilot  did  not  respond  to  the  question  or  that  the  response  was  illegible. 
Each  pilot  completed  his  or  her  own  survey,  resulting  in  a  total  of  sixteen  survey  forms  for  eight 

crews. 


Table  5-3a.  Flight  Time  (hours) 


POSITION 

TOTAL  FLIGHT  TIME 

MEAN 

RANGE 

Captain 

(n=8) 

107,000 

13,375 

11,000  to  18,000 

First  Officer 
(n=6) 

57,000 

9,500 

6,000  to  13,000 

Table  5-3b.  Hour  in  Type  (A320) 


POSITION 

TOTAL  FLIGHT  TIME 

MEAN 

RANGE 

Captain 

(n=8) 

11,100 

1,387 

150  to  2,500 

First  Officer 
(n=6) 

11,680 

1,946 

500  to  3,000 
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Survey  Items  1  through  4. 

Items  1  through  4  of  the  survey  refer  to  various  types  of  ATC-directed  breakouts  when 
using  autopilot  versus  when  autopilot  is  not  used.  For  each  of  these  items,  each  pilot  was  asked 
to  indicate  a  rating  by  circling  one  of  five  numbers  on  a  five-point  scale: 


1  2  3  4 


5 


Strongly  Disagree 


Neutral 


Strongly  Agree 


The  survey  items  are  listed  below. 


Survey  Item  1.  The  ATC-directed  climbing  breakout  is  easy  to  execute  when  using 
autopilot. 

Survey  Item  2.  The  ATC-directed  climbing  breakout  is  easy  to  execute  when  autopilot 
is  not  used. 


Survey  Item  3.  The  ATC-directed  descending  breakout  is  easy  to  execute  when  using 
autopilot. 

Survey  Item  4.  The  ATC-directed  descending  breakout  is  easy  to  execute  when 
autopilot  is  not  used. 

Figures  5-2  and  5-3  graphically  depict  the  ratings  of  ease  of  executing  ATC-directed 
breakouts  when  autopilot  is  used  versus  when  it  is  not  used  for  both  climbing  (Figure  5-2)  and 
descending  (Figure  5-3)  breakouts.  In  general,  the  responses  to  each  of  these  questions  were 
quite  similar.  Whether  the  ATC-directed  breakout  (climbing  or  descending)  was  executed  when 
using  autopilot  or  not,  the  majority  of  pilots  rated  the  breakout  as  generally  easy  to  perform. 


49 


Figure  5-2.  Ratings  of  ease  of  executing  ATC-directed  climbing  breakouts. 
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Figure  5-^3.  Ratings  of  ease  of  executing  ATC-^directed  descending  breakouts. 


Survey  Item  5.  More  crew  coordination  is  required  for  simultaneous  close  parallel 
approaches  than  for  normal  ILS  approaches. 

Each  pilot  was  asked  to  respond  by  using  the  same  5-point  scale  as  used  in  the  previous 
questions.  Fourteen  of  the  sixteen  pilots  (87.5%)  responded  "4"  or  "5"  indicating  that  they 
agreed  or  strongly  agreed  that  more  crew  coordination  is  required  for  simultaneous  close  parallel 
approaches  than  for  normal  ELS  approaches.  The  remaining  two  pilots  responded  "2"  indicating 
that  they  disagreed,  i.e.,  more  crew  coordination  is  not  required. 

Following  Items  1  through  5,  pilots  were  asked  to  comment  regarding  the  ATC-directed 
breakout  procedures.  Those  comments  were: 

•  Auto-throttles  slow  to  respond  to  open  climb  /  expedite  mode  on  climbing 
breakout. 

•  Climbing  breakout  using  autopilot  is  easier  and  quicker  above  400'  than 
below  400',  but  throttle  response  very  slow  when  left  in  climb. 

•  I  would  recommend  using  call  signs  at  the  end  of  the  "Alert"  breakaway 
command  due  to  blocked  transmission.  If  initial  call  sign  is  blocked,  either 
all  aircraft  will  breakaway  or  none  will. 

•  Autopilot  breakouts  to  me  were  easier  for  the  reason  there  is  less  verbal 
communication.  PF  (pilot  flying)  on  the  A320  is  operating  the  FCU. 

•  New  clear  command  for  breakout  may  be  a  good  idea  —  ATC  should  inform 
aircraft  that  simultaneous  close  parallel  approaches  are  in  progress. 

•  Changing  FCU  requires  extra  vigilance  especially  when  changing  from 
approach  to  HDGA^S  ('heading/vertical  speed).  Heading  selection  often 
changes  to  pre-set  heading  and  must  be  reset  and  confirmed  with  ATC. 

•  Not  well  suited  to  A320  automation. 

•  The  descending  breakout  goes  against  normal  instincts  as  does  the  "open 
climb."  I  want  to  go  to  TOGA. 

Survey  Items  6  through  11  concerned  the  use  of  various  types  of  information  sources  and 
training  materials.  Items  6  through  9  concerned  increasing  situational  awareness.  Item  10 
concerned  the  training  bulletins  and  whether  they  aided  in  increasing  understanding  of 
expectations  during  simultaneous  close  parallel  approaches.  Item  1 1  concerned  the  training 
bulletins  and  whether  they  helped  the  pilot  in  executing  the  ATC-directed  breakout.  Each 
question  is  listed  below.  Figure  5-4  graphically  illustrates  the  responses  to  each  of  the  questions. 

■Survey  Item  6.  The  approach  plate  notes  regarding  simultaneous  close  parallel 
approaches  increased  my  awareness  of  possible  traffic  in  close  proximity 
on  the  adjacent  approach. 

Survey  Item  7.  The  airport  information  page  increased  my  awareness  of  simultaneous 
close  parallel  approach  procedures. 
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Survey  Item  8. 


The  new  ATC  phraseology  ("Traffic  Alert")  coupled  with  the  word 
"immediately"  encouraged  me  to  respond  more  quickly  to  the  breakout 
maneuver  than  I  would  have  if  only  the  work  "immediately"  were  used. 


Survey  Item  9.  The  video  material  increased  my  awareness  of  simultaneous  close 
parallel  approach  operations. 

Survey  Item  10.  The  training  bulletins  increased  my  understanding  of  what  is  expected  of 
me  during  simultaneous  close  parallel  approaches. 

Survey  Item  11.  The  training  bulletins  helped  me  to  execute  the  ATC-directed  breakout 

As  shown  in  Figure  5-4,  pilots  generally  reported  an  increase  in  situational  awareness  due 
to  the  approach  plate  notes,  airport  information  page,  new  ATC  phraseology,  and  video.  Pilots 
also  generally  reported  the  positive  effects  of  the  training  bulletins.  Following  Items  6  through 
11,  each  pilot  was  asked  to  list  comments  regarding  the  material  provided  (approach  plate  notes, 
airport  information  page,  new  ATC  phraseology,  video,  and  the  training  bulletin).  Those 
comments  are  listed  below  and  are  categorized  by  topic: 

Approach  Plate  Notes 

•  Not  powerful  enough  language. 

Airport  Information  Page 

•  Airport  Information  Page  is  too  much  "fine  print"  -  would  rather  just 
see  a  clear  &  concise  note  on  Approach  Page. 

New  ATC  Phraseology 

•  A  lot  easier  to  do  the  2nd  or  3rd  time  than  the  first.  Good  wording  -  Traffic 
Alert  lets  you  know  instantly  what  the  breakout  is  for. 

•  Heightened  need  to  turn  quickly. 

•  Urgent  traffic  alert  should  be  different  from  TCAS  (Traffic  Alert  and 
Collision  Avoidance  System)  alerts. 

•  It  prepared  me  to  act  on  the  following  command. 

Video 

•  Video  was  informational  and  self  explaining. 

•  Good  video. 

•  Good  video. 
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Pe«:entage  of  Pilot  Responses  (n  =  1 8)  Percentage  of  Pilot  Responses  (n  =  1 8)  Percentage  of  Pilot  Responses  (n  =  18) 
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Figure  5-4.  Ratings  of  the  information  sources  and  training  materials. 
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Training  Bulletins 

•  I  would  strongly  recommend  against  doing  this  by  bulletin.  Hands-on  is 
best. 

•  Training  video  was  good,  made  me  more  aware  of  aircraft  locations  and 
distances. 

Other  Comments 

•  Good  material. 

•  The  material  prepared  me  for  the  heightened  awareness  necessary  for  the 
breakout. 

•  Simulation  training  should  be  required  for  this  procedure  during  normal 
SVT  (a  term  for  simulator  six-month  check). 

Survey  Item  12,  Based  on  the  information  presented,  how  would  you  choose  to  fly 
simultaneous  close  parallel  approaches  in  IMC  weather  with  CAT  I 
minima? 

Pilots  responded  by  selecting  from  the  following  choices: 

1  2  3 

Coupled  Autopilot  Hand  Flown/  No  Preference 

Flight  Director 

The  distribution  of  responses  is  seen  in  Figure  5-5.  Twelve  (75%)  of  the  sixteen  pilots 
chose  "coupled  autopilot"  Of  the  remaining  four  pilots,  three  (18.75%)  chose  "hand  flown/flight 
director"  and  one  (6.25%)  indicated  "no  preference." 


100^ 

9oJ 


1  2  3 

Coupled  Hand  Flown/  No 

Autopilot  Flight  Director  Preference 


Figure  5-5.  Mode  preference  for  flying  simultaneous  close  parallel  approaches  in 

IMC  weather  with  CAT  I  minima. 
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Survey  Item  13.  Based  on  your  experience  in  this  study,  how  would  you  choose  to 
execute  tiie  ATC-directed  breakout? 

Pilots  responded  by  selecting  from  the  following  choices: 


1 

2 

3 

Autopilot 

Hand  Flown 

No  Preference 

The  distribution  of  responses  is  seen  in  Figure  5-6.  Nine  of  the  sixteen  pilots  (56.25%) 
chose  "autopilot."  Six  pilots  (37.5%)  chose  "hand  flown"  and  one  pilot  (6.25%)  indicated  "no 
preference." 


Figure  5-6.  Mode  preference  for  executing  the  ATC-directed  breakout 


Survey  Item  14.  Should  Approach  Procedure  Charts  for  simultaneous  close  parallel 
approach  procedures  have  a  special  title  similar  to  Category  II/III 
approach  charts? 

Fourteen  (87.5%)  of  the  sixteen  pilots  responded  "yes."  One  pilot  (6.25%)  responded 
"no,"  and  one  pilot  (6.25%)  did  not  respond.  No  comments  were  given. 

Survey  Item  15.  What  additional  information  or  training  would  you  like  to  have  for 
simultaneous  close  parallel  approach  procedures? 

The  following  comments  were  made: 

•  Include  one  during  SVT  simulator  period. 

•  More  emphasis  on  who  operates  autopilot  when  given  breakaway. 

Emphasize  crew  coordination  and  duties. 
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•  Know  in  advance  the  likely  procedure  to  be  flown  in  the  event  of  a 
climbing/descending  breakout. 

•  Video  and  simulation  training. 

•  Maybe  one  standard  way  of  doing  the  approach. 

•  Procedures  for  breakoff  with  autopilot  "on"  as  going  to  TOGA  causes  loss 
of  heading  select. 

•  A320  breakout  above  400'  (no  TOGA)  slow  on  climb  performance. 

•  Procedures  specific  to  aircraft,  which  should  be  essentially  the  same  for  all 
a/c  at  NWA  (Northwest  Airlines). 

•  Would  need  simulation  training  for  approaches  and  missed  approaches. 

•  During  training  or  SVT  practice  a  few  parallel  approaches. 

•  ATC  should  advise  each  aircraft  of  close  approaches  prior  to 
commencement. 

Survey  Item  16,  Additional  comments: 

•  I  am  uncomfortable  with  the  400'  call  and  additional  duties  associated  with 
it.  How  about  all  breakaways  be  done  at  TOGA  power? 

•  I  am  more  comfortable  using  TOGA  power  on  ^  climbing  breakouts 
regardless  of  above  or  below  400! 

•  On  A320  400'  (part  of  training  procedure)  and  200'  above  minimum 
(decision  altitude)  call  is  simultaneous  -  this  is  a  problem  that  would  have  to 
be  worked  out  if  this  system  becomes  reality. 

•  400'  call-out  is  same  altitude  as  200'  above  minimum  (decision  altitude)  for 
CAT  I.  This  becomes  confusing  -  most  crews  will  miss  one  or  the  other. 

•  For  A320  and  other  automated  a/c  need  to  specify  which  pilot  will  operate 
"panel"  during  autopilot  approach. 

•  Good  period,  thanks! 

•  The  A3 20  should  go  to  TOGA  power.  The  aircraft  was  designed  that  way. 

5.3  SUMMARY  OF  PHASE  1  AND  PHASE  2  SURVEY  RESULTS 

In  Phase  1,  pilots  received  no  special  training  or  materials  to  make  them  aware  that 
closely  spaced  simultaneous  approaches  were  in  operation.  They  were  told  to  fly  the  approach  in 
the  simulation  as  they  normally  would  fly  during  actual  conditions.  In  Phase  2  pilots  were  given 
specific  training  and  informational  materials  regarding  simultaneous  approaches  to  closely 
spaced  parallel  runways. 

Results  of  Phase  1  indicated  the  need  for  specific  training  and  materials  to  heighten  the 
awareness  of  pilots  regarding  the  conduct  of  simultaneous  approaches  to  closely  spaced  parallel 
runways.  In  Phase  1  Survey  Item  10,  responses  indicated  that  most  crews  reported  developing  a 
strategy  for  making  a  decision  to  turn  and  then  to  execute  the  turn.  This  indicates  the  potential 
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for  training  on  the  procedure  to  affect  future  performance.  Phase  1  Survey  Item  11  responses 
indicated  that  90%  of  the  crews  thought  that  training  or  better  situational  awareness  would  have 
enhanced  performance  during  the  ATC-directed  breakout.  Comments  to  many  of  the  questions 
indicated  that  training  involving  awareness  of  the  situation  and  practice  through  the  use  of 
simulation  would  be  desirable.  These  findings  confirmed  the  need  for  A320  Phase  2,  in  which 
specific  training  and  materials  were  provided  to  heighten  situational  awareness  and  to  potentially 
^fect  pilot  performance. 

Responses  to  study  Phase  2  Survey  Item  5  indicated  that  the  majority  of  pilots  felt  that 
more  crew  coordination  is  required  for  simultaneous  close  parallel  approaches  than  for  normal 
ILS  approaches.  In  the  Phase  2  Survey,  Items  6  through  11  concerned  the  use  of  various  types 
of  information  sources  and  training  materials  and  their  ability  to  help  in  increasing  situational 
awareness.  Figures  5-2  and  5-3  show  that  the  majority  of  , responses  indicated  that  pilots 
"agree"  or  "strongly  agree"  that  awareness  is  increased  by  use  of  informational  materials, 
including:  approach  plate  notes,  airport  information  page,  video  material,  and  training  bulletins. 
Pilot  responses  also  indicated  that  the  new  ATC  phraseology  used,  i.e.,  "Traffic  Alert"  coupled 
with  the  word  "immediately,"  helped  to  foster  a  quick  response. 

Phase  2  Survey  responses  indicated  that  75%  of  pilots  would  choose  to  fly  simultaneous 
close  parallel  approaches  in  MC  weather  CAT  I  minima  using  coupled  autopilot  rather  than 
hand  flown/flight  director.  Based  on  their  experience  in  Phase  II,  the  majority  of  pilots  chose 
autopilot  versus  hand  flown  in  executing  the  ATC-directed  breakout. 
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6.  DISCUSSION 


6.1  RESEARCH  QUESTIONS 

The  testable  questions  posed  in  Section  1 .2  asked  about  the  A320  breakout  performance 
in  each  phase  and  how  the  turn  performance  compared  to  that  observed  in  the  DCIO  and  B727 
studies.  These  questions  were  further  refined  into  the  nine  research  questions  listed  in 
Section  2.3. 

Research  questions  1  and  5  asked  if  either  the  Phase  1  or  Phase  2  turn  performance  met 
the  test  criteria  for  mean  and  maximum  time  from  start  of  ATC  breakout  instruction  to  start  of 
turn.  Except  for  two  combinations  of  approach  mode  and  breakout  group,  Phase  1  mean  values 
were  greater  than  the  8-second  criterion.  In  addition,  twelve  percent  of  the  times  to  start  of  turn 
were  greater  than  the  17-second  criterion.  The  data  in  Table  4-11  indicate  that  the  long  response 
times  were  not  limited  to  one  approach  mode  or  to  one  breakout  group.  In  Phase  2,  mean  time  to 
start  of  turn  was  less  than  8  seconds  for  all  combinations  of  approach  mode  and  breakout  group. 
A  comparison  between  the  Phase  1  distributions  in  Figure  4-4  and  the  Phase  2  distributions  in 
Figure  4-5  indicated  that  the  smaller  mean  values  in  Phase  2  were  mainly  due  to  the  reduction  in 
the  frequency  of  long  response  times  rather  than  a  general  decrease  in  response  times.  Although 
the  number  of  long  response  times  decreased  in  Phase  2,  there  were  three  trials  with  times  to  start 
of  turn  of  greater  than  17  seconds.  Two  factors  were  associated  with  these  three  trials,  breakouts 
at  decision  altitude  and  individual  pilot  performance.  The  pilot  associated  with  two  of  the  long 
response  times  was  also  associated  with  seven  of  the  long  response  times  in  Phase  1. 

Statistical  comparisons  between  the  Phase  1  and  Phase  2  turn  data  confirmed  that  the 
smaller  mean  time-to-tum  values  in  Phase  2  were  not  due  to  a  shift  in  the  response  distributions. 
The  analysis  of  variance  results  indicated  that  the  mean  values  were  not  significantly  different 
between  the  two  phases. 

Research  questions  2  and  3  asked  about  pilot  comfort  executing  climbing  breakouts 
below  400  feet  AGL  and  descending  breakouts  outside  the  outer  marker  in  Phase  1.  The  pilots 
rated  these  breakouts  as  being  "not  at  all"  to  "moderately"  difficult.  The  distributions  of 
responses  were  similar  to  those  for  climbing  breakouts  inside  and  outside  the  outer  marker.  The 
ratings  were  also  similar  for  hand-flown  and  autopilot-coupled  approach  modes. 

Research  question  6  asked  which  situational  awareness  aids  were  helpful  to  the  pilots  in 
Phase  2.  At  least  88  percent  of  the  responses  were  "neutral"  to  "strongly  agree"  for  each  of  the 
four  aids:  approach  plate  modifications;  airport  information  page;  video  material;  and,  breakout 
training  bulletin.  Ninety-two  percent  agreed  that  the  new  ATC  phraseology  was  helpful.  Of 
these,  seventy-five  percent  strongly  agreed  that  the  new  ATC  phraseology  encouraged  them  to 
respond  more  quickly  than  the  current  phraseology. 

Research  questions  4,  7,  8,  and  9  asked  if  any  of  the  test  conditions  (independent 
variables)  affected  breakout  performance.  Similar  results  were  observed  in  both  phases  of  the 
study.  First,  altitude  affected  both  climb  and  turn  performance,  although  the  trends  were 
opposite.  Mean  times  from  start  of  ATC  instruction  to  start  of  engine,  pitch,  and  vertical  speed 
responses  were  smallest  at  decision  altitude  and  increased  with  increasing  altitude.  Similarly, 
mean  altitude  loss  was  smallest  at  decision  altitude,  and  increased  with  increasing  altitude.  In 
contrast,  mean  time  from  start  of  ATC  instruction  to  start  of  turn  was  largest  at  decision  altitude 
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but  similar  for  the  500',  1000',  and  1800'  scenarios.  Second,  approach  mode  affected  both  turn 
and  climb  performance.  In  general,  the  measured  dependent  variables  were  smaller  for  hand- 
flown  scenarios  than  for  autopilot-coupled  scenarios.  Based  on  the  data  in  Figures  4-4  and  4-5, 
median  time  to  start  of  turn  was  1  to  3  seconds  less  for  hand-flown  scenarios  at  altitudes  above 
400  feet  AGL  and  6  seconds  less  for  hand-flown  scenarios  at  decision  altitude.  Third,  direction 
of  the  turn  did  not  affect  turn  performance  in  either  phase.  Mean  times  to  start  of  turn  were 
similar  for  climbing  and  descending  breakouts  at  1800  feet  AGL. 

The  engine-out  scenarios  were  intended  to  measure  the  effect  of  cockpit  distractions  on 
A320  breakout  performance.  Instead,  they  tested  breakout  performance  during  an  aircraft 
emergency.  During  six  engine  out  trials  (30  percent),  the  pilot  flying  elected  to  continue  straight 
ahead  rather  than  turn.  This  result  indicates  that  these  scenarios  were  not  appropriate  as 
distractors  and  that  a  more-common  distraction  scenario  should  be  developed  for  future  studies. 

When  only  these  trials  in  which  the  crew  complied  with  the  ATC  instruction  were 
considered,  the  engine-out  event  in  Phase  1  had  little  or  no  effect  on  breakout  performance  at 
decision  altitude.  Mean  times  to  start  of  turn  and  to  TOGA  were  statistically  similar  for  the  two 
levels  of  distraction.  However,  mean  times  to  engine  rpm  increase  and  to  start  of  climb  were 
significantly  smaller  for  the  engine-out  scenarios.  These  shorter  response  times  may  have  been  a 
result  of  the  pilots  responding  to  the  engine  failure,  which  occurred  before  the  breakout 
instruction,  rather  than  a  response  to  the  breakout  instruction. 

6.2  ANCILLARY  ISSUES 

Survey  questions  elicited  pilot  opinion  regarding  other  aspects  of  ATC-directed 
breakouts.  Based  on  their  experience  in  Phase  1,  the  subjects  pilots  felt  there  was  a  need  for 
training  and  heightened  situational  awareness.  Six  crews  felt  their  response  would  have  been 
different  at  low  altitude  if  they  had  received  the  additional  training.  Eight  crews  replied  that  they 
developed  a  strategy  for  executing  the  breakout.  These  responses  suggest  that  practicing  a 
specific  breakout  procedure  would  increase  pilot  comfort  with  the  maneuver.  Phase  2  subjects 
pilots  also  commented  on  the  need  for  simulator  training. 

When  asked  what,  if  any,  aircraft  limitations  they  thought  caused  an  inherent  unwanted 
delay,  the  most  frequent  response  was  the  problem  with  TOGA  thrust  reverting  to  the  go-around 
track.  These  pilot  observations  agreed  with  the  observed  rollbacks  in  the  aircraft  data.  In 
fourteen  autopilot-coupled  trials,  the  rollback  lasted  long  enough  to  add  4  to  18  seconds  to  the 
turn  response. 

In  Phase  2,  some  subject  pilots  commented  on  the  tested  breakout  procedure.  Several  did 
not  like  the  altitude  callout  at  400  feet  AGL  because  it  interfered  with  the  callout  procedure  at 
200  feet  above  minimums.  There  were  also  comments  about  using  open  climb  instead  of  TOGA 
thrust.  One  concern  was  that  leaving  the  thrust  levers  at  the  Climb  detent  is  counter-intuitive 
because  the  pilots  are  trained  to  use  TOGA  thrust.  The  other  concern  was  the  sluggish 
autothrottle  response  during  open  climb.  The  latter  concern  agreed  with  the  observed  climb 
performance.  Mean  time  difference  between  pitch  increase  and  vertical  speed  increase  was 
longer  in  Phase  2  than  in  Phase  1.  In  addition,  the  mean  time  to  transition  from  a  negative  to  a 
positive  climb  rate  was  longer  in  Phase  2.  In  Phase  1,  TOGA  thrust  was  used  in  88  percent  of 
the  climbing  breakouts  above  400  feet  AGL,  whereas  TOGA  thrust  was  used  in  only  24  percent 
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of  the  climbing  breakouts  above  400  feet  AGL  in  Phase  2.  Open  climb  mode  was  used  in  the 
remainder  of  the  trials. 

It  is  not  known  if  a  delay  in  climb  response  would  affect  the  safety  of  close  parallel 
approach  operations.  The  emphasis  to-date  has  been  on  turn  performance,  and  little  is  known 
about  the  effect  of  climb  performance  on  blunder  safety  analyses.  We  recommend  further 
research  on  the  role  of  climb  response  in  breakout  performance. 

6.3  CONCLUSIONS 

Three  conclusions  were  made  concerning  A320  breakout  performance  in  this  study. 
First,  turn  performance  in  Phase  1  did  not  meet  the  best  criteria  for  either  manual  or  autopilot- 
coupled  approach  modes.  This  suggests  that  the  existing  combination  of  pilot  training  and  ATC 
phraseology  is  not  sufficient  to  guarantee  a  satisfactory  distribution  of  A320  breakout  responses. 

Second,  turn  performance  in  Phase  2  did  meet  the  test  criteria  for  most  combinations  of 
breakout  group  and  approach  mode.  This  suggests  that  the  proposal  ATC  breakout  phraseology 
combined  with  pilot  situational  awareness  training  and  a  breakout  procedure  is  sufficient  to 
achieve  a  satisfactory  distribution  of  A320  breakout  responses.  Mean  time  to  start  of  turn  was 
less  than  8  seconds  for  all  breakout  groups  and  approach  modes.  There  were  3  times  to  start  of 
turn  greater  than  17  seconds.  However,  the  effect  of  this  small  percentage  could  be  mitigated  if 
breakouts  were  limited  below  400  feet  AGL.  The  results  also  illustrated  that  some  pilots  will 
respond  more  slowly  than  expected  even  with  increased  situational  awareness. 

Third,  the  tested  breakout  procedure  should  be  refined  because  it  conflicted  with  other 
cockpit  procedure  and  increased  the  average  time  to  transition  from  a  negative  to  a  positive 
climb  rate.  The  pilots  were  trained  to  use  TOGA  thrust,  and  were  not  comfortable  with  leaving 
the  thrust  levers  at  the  climb  detent.  They  also  were  not  comfortable  with  the  altitude  callout  at 
400  feet  AGL.  In  addition,  open-climb  performance  was  sluggish  at  higher  altitudes. 

Based  on  the  above  conclusions,  it  is  recommended  that  a  combination  of  pilot  awareness 
training,  A320  breakout  procedure,  and  ATC  breakout  phraseology  equivalent  to  that  used  in 
Phase  2  be  employed  for  simultaneous  parallel  approach  operations  in  instrument  meteorological 
conditions.  The  implemented  A320  breakout  procedure  should  account  for  the  effects  of  thrust 
management  (TOGA  or  open  climb)  on  turn  and  climb  performance  during  autopilot-coupled 
breakouts. 
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GLOSSARY 


A320 

Airbus  320 

AGL 

above  ground  level 

ALPA 

Air  Line  Pilots  Association 

ALT 

altitude 

ANOVA 

analysis  of  variance 

APor  A/P 

autopilot 

AP/AP 

autopilot  approach  /  autopilot  breakout 

AP/HF 

autopilot  approach  /  hand-flown  breakout 

ARTS 

Automated  Radar  Terminal  System 

ATC 

Air  Traffic  Control 

ATIS 

Automatic  Terminal  Information  System 

Bin 

Boeing  727 

B747-400 

Boeing  747-400 

CAT 

category 

CL 

climb 

DA 

decision  altitude 

DCIO 

McDonnell  Douglas  10 

DH 

decision  height 

DME 

Distance  Measuring  Equipment 

FAA 

Federal  Aviation  Administration 

FAR 

Federal  Aviation  Regulations 

FCU 

flight  control  unit 

FDorF/D 

flight  director 

FMS 

flight  management  system 

G  A  or  G/A 

go  around 

GS 

glide  slope 

GATRK 

go  around  track 

HDG 

heading 

HDG  SEL 

heading  select 

HF 

hand-flown 

HF/HF 

hand-flown  approach  /  hand-flown  breakout 

ILS 

instrument  landing  system 
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IMC 

instrument  meteorological  conditions 

LNAV 

localizer  navigation  mode 

LOG 

localizer 

MA 

missed  approach 

MCDU 

multipurpose  cockpit  display  unit 

MD80 

McDonnell  Douglas  80 

MEM 

Memphis  International  Airport 

MPAP 

Multiple  Parallel  Approach  Procedure 

MVA 

minimum  vectoring  altitude 

NATCO 

Northwest  Aerospace  Training  Corporation 

NAV 

navigation 

ND 

navigational  display 

NOS 

National  Ocean  Service 

NTZ 

no  transgression  zone 

NWA 

Northwest  Airlines 

PF 

pilot  flying 

PFD 

primary  flight  display 

PNF 

pilot  not  flying 

PRM 

Precision  Runway  Monitor 

SVT 

Simulator  training 

TCAS 

Traffic  Alert  and  Collision  Avoidance  System 

TOGA  or  TO/GA 

take  off/go  around  thrust 

TRACON 

Terminal  Radar  Approach  Control 

TWG 

technical  working  group 

UAL 

United  Airlines 

VMC 

visual  meteorological  conditions 

VNAV 

vertical  navigation  mode 

VS 

vertical  speed 
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APPENDIX  A.  SCENARIO  SEQUENCES 


The  following  sub-sections  list  the  information  for  each  test  scenario  in  Phases  1  and  2  in 
the  order  in  which  they  were  conducted.  Breakout  DME  was  the  range  (nautical  miles)  from  the 
runway  threshold  at  which  the  air  traffic  controller  started  the  breakout  instruction.  Breakout 
heading,  direction,  and  altitude  were  the  values  used  in  the  Air  Traffic  Control  breakout 
phraseology.  Simulator  scenario  was  the  actual  weather  condition  entered  into  the  simulator, 
while  the  weather  card  was  the  weather  reported  to  the  air  crew  for  that  scenario.  The  actual  and 
reported  weather  information  are  listed  in  Table  A-1. 

In  Phase  1,  the  approaches  were  either  hand-flown  with  the  flight  director  on  (HF)  or 
using  the  autopilot  coupled  to  the  flight  director  (AP).  Whether  or  not  the  breakout  was 
conducted  using  the  autopilot  was  at  the  discretion  of  the  pilot  flying.  All  HF  approaches  were 
Category  I  and  all  AP  approaches  were  Category  n.  The  scenario  sequence  was  the  same  for  all 
subject  crews.  The  subjects  were  not  assigned  a  priori  to  each  scenario. 

In  Phase  2,  the  approaches  were  either  hand-flown  with  the  flight  director  on  (HF)  or 
using  the  autopilot  coupled  to  the  flight  director  (AP).  Whether  or  not  the  breakout  was 
conducted  using  the  autopilot  was  at  the  discretion  of  the  pilot  flying.  All  approaches  were 
Category  I  except  for  an  autopilot  breakout  at  decision  altitude  and  an  autopilot  missed  approach 
(MA).  Pilot  flying  was  assigned  to  either  the  captain  (C)  or  first  officer  (F).  Scenario  order  and 
pilot  assignments  were  rotated  among  the  four  sequence  sheets  listed  in  Section  A.2. 


Table  A-1.  Weather  Conditions 


Simulator  Scenario; 

001 

300  overcast,  visibility  1 ,  wind  350/5 

(actual  weather) 

002 

Zero/Zero  visibility,  winds  350/5 

003 

Zero/Zero  visibility,  winds  270/15 

Weather  Card: 

1 

300  overcast,  visibility  1 ,  wind  350/5,  altimeter  29.92 

(reported  weather) 

2 

100  overcast,  visibility  1/4,  wind  350/5,  altimeter  29.92 

3 

200  overcast,  visibility  1/2,  wind  270/15,  altimeter  29.92 
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A.1  PHASE  1 


Table  A-2.  A320  Phase  1  Run  Sequence 


Scenario 

Number 

Approach 

Mode 

Simulator 

Scenario 

Weather 

Card 

Breakout 

DME 

Breakout 

Heading 

Breakout 

Direction 

Breakout 

Altitude 

100 

HF 

001 

1 

lands 

— 

— 

— 

111 

HF 

002 

1 

1.7 

300 

Climb 

3500 

116 

HF 

003 

3 

5.6 

285 

Descend 

1800 

105 

AP 

001 

1 

lands 

— 

103 

AP 

002 

2 

0.5 

290 

Climb 

3000 

*108* 

HF 

005 

1 

0.8 

290 

Climb 

3000 

109 

HF 

003 

3 

5.6 

280 

Climb 

4000 

100 

HF 

001 

1 

lands 

— 

— 

-- 

115 

HF 

003 

3 

5.6 

285 

Climb 

4200 

113 

HF 

003 

3 

0.8 

285 

Climb 

3000 

112 

HF 

002 

1 

MA 

— 

— 

-- 

110 

AP 

002 

2 

MA 

— 

106 

AP 

002 

2 

1.8 

300 

Climb 

3000 

*107* 

AP 

004 

2 

0.5 

270 

Climb 

4000 

117 

AP 

002 

2 

5.6 

300 

Descend 

1800 

104 

AP 

002 

2 

5.6 

290 

Climb 

5000 

100 

HF 

001 

1 

lands 

1 

1 

1 

1 

1 

1 

1 

1 

114 

AP 

002 

2 

1.7 

280 

Climb 

4000 

103 

AP 

002 

2 

0.5 

290 

Climb 

3000 

113 

HF 

003 

3 

"  0.8 

285 

Climb 

3000 

101 

AP 

002 

2 

5.6 

280 

Climb 

4000 

102 

HF 

003 

3 

1.8 

290 

Climb 

3700 

*107  and  108  included  right  engine  failure  shortiy  before  breakout. 
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A.2  PHASE  2 


Table  A-3.  A320  Phase  2  Sequence  1  (Crews  1, 5, 9) 


Approach 

Mode 

■dlfgM 

liraSI 

Simulator 

Scenario 

Weather 
Card  . 

Breakout 

DME 

Breakout 

Direction 

Breakout 

Altitude 

AP 

002 

1 

6.0 

280 

Climb 

4000 

HF 

rr- 

002 

1 

2.0 

300 

Climb 

3500 

lands 


6.0 


3.3 


1.9 


6.3 


6.1 


0.9 


1.9 


MA 


6.2 


1.9 


0.6 


5.8 


MA 


1.9 


3.5 


5.9 


lands 


1.8 


5.9 


Table  A-4.  A320  Phase  2  Sequence  2  (Crews  2, 6) 


Scenario 

Number 


201 


211 


200 


220 


218 


206 


216 


209 


213 


221 


212 


217 


222 


*203* 


215 


*210* 


214 


219 


223 


205 


202 


204 


Approach 

Mode 


AP 


HF 


HF 


AP 


AP 


AP 


HF 


HF 


HF 


AP 


HF 


AP 


AP 


AP 


HF 


AP 


AP 


HF 


AP 


AP 


HF 


AP 


Simulator 

Scenario 

Weather 

Card 

Breakout 

DME 

002 

1 

6.0 

002 

1 

2.0 

001 

1 

lands 

002 

1 

6.0 

002 

1 

3.3 

002 

1 

1.9 

003 

3 

6.3 

003 

3  - 

6.1 

003 

3 

0.9 

003 

3 

1.9 

002 

1 

MA 

002 

1 

6.2 

003 

3 

1.9 

002 

2  n 

0.6 

003 

3 

5.8 

002 

2 

MA 

002 

1 

1.9 

003 

3 

3.5 

002 

1 

5.9 

001 

1 

lands 

003 

3 

1.8 

002 

1 

5.9 
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Table  A-5.  A320  Phase  2  Sequence  3  (Crews  3, 7, 10) 


Scenario 

Number 

Approach 

Mode 

liRI 

Simulator 

Scenario 

Weather 

Card 

Breakout 

DME 

Breakout 

Heading 

Breakout 

Direction 

Breakout 

Altitude 

204 

AP 

002 

1 

5.9 

290 

Climb 

5000 

202 

HF 

c 

003 

3 

1.8 

290 

Climb 

3700 

205 

AP 

F 

001 

1 

lands 

— 

— 

223 

AP 

F 

002 

1 

5.9 

280 

Climb 

4000 

219 

HF 

F 

003 

3 

3.5 

285 

Climb 

3000 

214 

AP 

C 

002 

1 

1.9 

280 

Climb 

4000 

*210* 

AP 

F 

002 

2 

MA 

— 

— 

— 

215 

HF 

C 

003 

3 

5.8 

285 

Climb 

4200 

*203* 

AP 

F 

002 

2 

0.6 

290 

Climb 

3000 

222 

AP 

F 

003 

3 

1.9 

290 

Climb 

4000 

217 

AP 

C 

002 

1 

6.2 

300 

Descend 

1500 

212 

HF 

C 

002 

1 

MA 

— 

-- 

— 

221 

AP 

F 

003 

3 

1.9 

270 

Climb 

3000 

213 

HF 

C 

003 

3 

0.9 

285 

Climb 

3000 

209 

HF 

F 

003 

3 

6.1 

280 

Climb 

3500 

216 

HF 

F 

003 

3 

6.3 

285 

Descend 

1400 

206 

AP 

C 

002 

1 

1.9 

300 

Climb 

3000 

218 

AP 

C 

002  1 

1 

3.3 

285 

Climb 

3500 

220 

AP 

F 

002 

1 

6.0 

300 

Climb 

4000 

200 

HF 

C 

001 

1 

lands 

-- 

— 

— 

211 

HF 

F 

002 

1 

2.0 

300 

Climb 

3500 

201 

AP 

C 

002 

1 

6.0 

280 

Climb 

4000 

Table  A-6.  A320  Phase  2  Sequence  4  (Crews  4, 8) 


Simulator  Weather 
Scenario  Card 


002 


003 


001 


002 


003 


002 


002 


003 


002 


003 


002 


002 


003 


003 


003 


003 


002 


002 


002 


001 


002 


002 


Scenarios  203  and  210  were  Category  II  approaches. 


Scenario 

Number 

Approach 

Mode 

204 

AP 

202 

HF 

205 

AP 

223 

AP 

219 

HF 

214 

AP 

*210* 

AP 

215 

HF 

*203* 

AP 

222 

AP 

217 

AP 

212 

HF 

221 

AP 

213 

HF 

209 

HF 

216  n 

HF 

206 

AP 

218 

AP 

220 

AP 

200 

HF 

211 

HF 

201 

AP 

Breakout 

DME 


5.9 


1.8 


lands 


5.9 


3.5 


1.9 


MA 


5.8 


0.6 


1.9 


6.2 


MA 


1.9 


0.9 


6.1 


6.3 


1.9 


3.3 


6.0 


lands 


2.0 


6.0 


Breakout 

Direction 


Climb 


C 


Climb 


Climb 


Climb 


Descend 


Climb 


Ciimb 


Climb 


Descend 


Climb 


Climb 


Climb 


Climb 


Breakout 

Altitude 


5000 


3700 
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APPENDIX  B.  PHASE  1  APPROACH  PLATES 


Arndt  n  94174 

IIS  RWY  36L 


292 
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MEMPHIS  INTL  (MEM) 
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lOHN  pID 
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i 
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^^^356*  4.2  NM 
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IL$  RWY  36L 
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ii'.wi’eK^MKCTiFiamairgi 
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294 


Arndt  119X174  (CAT  II) 

ILS  RWY  36L 


AL-253  (FAA) 


SE-1, 13  OCT  1994 
AAEMPHIS  INTI  (MEM) 
MEMPHIS,  TENNESSEE 


CATEGORY  li  ILS-SPECIAL  AIRCREW 
&  AIRCRAFT  CERTIFICATION  REQUIRED 
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MEMPHIS,  TENNESSEE 
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APPENDIX  C.  PHASE  2  APPROACH  PLATES 


JEPPES3M 


6  MAY  94 


ATIS  127.75 

MtAAPHIS  Approach  (R)  354*- 175*  125.8 
176V355*  1  19.1 

MEMPHIS  Tov/*r  Rwyi  181-36R.I8R-361.  119.7 
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FREAZ 
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APPENDIX  D.  PHASE  2  WRITTEN  TRAINING  MATERIALS 


The  briefing  to  the  subject  crews  before  each  session  in  Phase  2  included  material  that 
presented  information  about  close  parallel  simultaneous  ILS  approaches.  Each  crew  was 
required  to  view  a  short  video  [11]  which  showed  the  Precision  Runway  Monitor  radar  and 
controller  displays  used  to  monitor  these  approaches.  The  video  also  discussed  possible 
interventions  by  the  monitor  controller,  and  emphasized  the  need  for  pilot  compliance  with  any 
breakout  instructions. 

After  viewing  the  video,  each  subject  pilot  was  required  to  read  an  1 1-0  Information  Page 
and  a  pilot  awareness  training  bulletin,  both  of  which  were  designed  for  this  test.  After  reading 
the  training  bulletin,  each  pilot  took  a  self-administered  (i.e.,  open-book)  test  which  reinforced 
the  important  concept  presented  in  the  bulletin.  The  pilots  were  not  graded  on  the  test. 

The  subject  crews  also  received  a  written  breakout  procedure  designed  specifically  for 
the  A320.  The  crews  read  the  procedure,  then  took  a  second  self-administered  test  which  was 
designed  to  reinforce  the  concepts  presented  in  the  written  procedure  bulletin.  The  crews  were 
required  to  follow  the  breakout  procedure  during  the  test  session.  There  was  no  other  procedure 
training,  and  the  pilots  did  not  practice  the  breakout  procedure  before  the  test  session.  The  test 
personnel  were  directed  to  not  provide  any  information  about  how  to  execute  the  procedure. 

The  following  pages  present  the  written  training  package  in  the  following  order:  the  11-0 
Information  Page;  the  Pilot  Awareness  Training  Bulletin;  the  Pilot  Awareness  Open  Book  Test; 
the  A320  Breakout  Procedure  Training  Bulletin;  and,  the  A320  Breakout  Procedure  Open  Book 
Test. 
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D.l  11-0  INFORMATION  PAGE 


(INFORMATION  PAGE) 
11-0 


MEMPHIS, TENN 


_ MEMPHIS  INTL 

*******FOR  SIMULATION  PURPOSES  ONLY**** 

PRECISION  RUNWAY  MONITORING 
RUNWAYS  36  L/R 

SIMULTANEOUS  CLOSE  PARALLEL  APPROACHES 

FAA  Order  71 10.65H  paragraph  5-127  authorizes  simultaneous  ILS  approaches  to  parallel  runways  with  centerlines 
separated  by  a  minimum  of  3400’.  The  previous  standard  was  4300’.  To  qualify  for  reduced  separation,  parallel 
runways  must  be  serviced  by  high  update  radar  and  high  resolution  ATC  radar  displays  collectively  called  a 
Precision  Runway  Monitor  (PRM).  MEM  runways  36  L/R  centerlines  are  separated  by  3400’,  and  are  served  by 
PRM,  permitting  simultaneous  ILS  approaches. 

The  enhanced  PRM  system  provides  controllers  almost  instantaneous  radar  information.  The  supporting  automated 
tracking  software  furnishes  the  controller  with  aircraft  identification  and  position,  a  ten  second  projected  position,  as 
well  as  visual  and  aural  alerts.  These  alerts  signal  the  controller  when  an  aircraft  deviates  off  the  localizer  towards 
the  “No  Transgression  Zone”  (NTZ)  between  final  approach  courses.  The  NTZ  standard  width  for  all  simultaneous 
parallel  approaches  (all  locations)  is  2000’.  It  is  established  equidistant  between  final  approach  courses  and  is 
depicted  on  the  controller’s  monitor  display. 

If  an  aircraft  is  observed  to  be  on  a  track  that  is  left/right  of  the  final  approach  course  and  may  penetrate  the  NTZ, 
the  controller  will  provide  instructions  to  return  the  aircraft  to  the  final  approach  course. 

Phraseology: 

“You  have  crossed  the  final  approach  course.  Turn  (left/right)  IMMEDIATELY  and  return  to  localizer 
azimuth/course. 

or 

“Turn  (left/right)  and  return  to  the  localizer/azimuth  course”. 

If  an  aircraft  is  observed  penetrating  the  NTZ,  ATC  instructions  will  be  given  to  the  aircraft  on  the  adjacent  final 
approach  course  to  alter  course  to  avoid  the  deviating  aircraft. 

Phraseology: 

“(Aircraft  callsign)  TRAFFIC  ALERT  (aircraft  callsign)  turn  (left/right)  IMMEDIATELY  heading  (degrees). 
(Climb/desc)end  and  maintain  (altitude)”. 

An  immediate  pilot  response  is  expected  and  required  to  avoid  the  imminent  situation.  Execution  of  these  ATC 
instructions  must  be  as  rapid  as  practical.  This  maneuver  may  be  performed  either  manually  or  on  autopilot,  m 
compliance  with  aircraft/company  operating  procedures. 

Simultaneous  ILS  approaches  are  authorized  for  both  parallel  runways  at  Memphis  International  Airport  and  will  be 
conducted  when  conditions  such  as  weather  and  traffic  flow  dictate.  The  ATIS  broadcast  will  advise  pilots  when 
closely  spaced  simultaneous  ILS  approaches  are  in  progress. 

For  questions,  please  contact:  Supervisor,  Memphis  TRACON,  1-800-555-1212.  _ | 
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FOR  SIMULATION  PURPOSES  ONLY 
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D.2  PILOT  AWARENESS  TRAINING  BULLETIN 

PILOT  AWARENESS 
TRAINING  BULLETIN 

Pilot  AWARENESS  Training  for  Simultaneous  Close  Parallel  ILS  Approaches 


At  180  kts.  an  aircraft  that  has  entered  the  No  Transgression  Zone  can  cross  the  adjacent  parallel 
course  centerline  in  as  little  as  NINE  SECONDS.  Inattention,  or  failure  to  expeditiously  comply 
with  a  final  monitor  controller's  breakout  instructions  could  result  in  a  midair  collision. 
Remember,  you  are  being  broken  off  the  ILS  because  an  aircraft  firom  the  adjacent  ILS  has 
probably  deviated  off  course  and  is  HEADING  YOUR  WAY.  When  pilots  hear  or  read  the 
word  "CLOSE"  in  association  with  simultaneous  parallel  approaches,  they  should  be  especially 
aware  that  any  instructions  issued  by  a  controller  should  be  immediately  followed. 

This  Flight  Operations  Bulletin  imparts  important  information  necessary  for  pilots  to  attain  the 
increased  level  of  pilot  awareness  required  for  safe,  efficient,  simultaneous  close  parallel  ILS 
approach  operations.  Important  pilot  questions  of  "why,"  "how  will  I  know,"  "what  can  I 
expect,"  and  "what  will  I  do"  are  answered.  Hopefully,  "SIMULTANEOUS  CLOSE 
PARALLEL  ILS  APPROACHES"  will  be  put  on  your  list  of  important  aviation  terms  or 
"buzzwords." 


WHY? 

Increased  crew  awareness  is  necessary  because  in  all  probability,  there  is  an  aircraft  operating  on 
the  adjacent  parallel  localizer  course  as  close  as  3400’  from  your  wingtip.  Failure  to  comply  with 
ATC  clearances,  tune  the  proper  localizer  frequency,  accurately  track  the  localizer  course 
centerline,  or  respond  to  controller  breakout  instructions  in  an  expeditious  manner  are  all  factors 
that  may  lead  to  loss  of  lateral  separation,  near-midair  collisions,  or  midair  collisions.  Attention 
to  detail  is  mandatory! 


HOW  WILL  I  KNOW? 

Key  words  such  as  "simultaneous"  and  "close  parallel"  should  alert  pilots  of  the  need  to 
increase  their  awareness  level.  ATIS  will  broadcast  phraseology  such  as  "Simultaneous  Close 
Parallel  Approaches  to  RWYS  (number)  L/R  in  use".  For  a  new  approach  procedure  the 
approach  plate  for  close  parallel  approaches  will  be  titled  "CLOSE  PARALLEL  ILS  RWY 
(number)L/R".  For  existing  approach  procedures  the  approach  plate  will  contain  the  note 
"SIMULTANEOUS  CLOSE  PARALLEL  APPROACHES  AUTHORIZED  WITH  RUNWAYS 
(number)  L/R",  and  "GLIDESLOPE  REQUIRED". 
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TRAINING  BULLETIN 

WHAT  CAN  I  EXPECT? 

If  you  or  the  aircraft  on  the  adjacent  localizer  course  fail  to  track  the  localizer  course  centerline 
(or  worse  yet,  track  the  wrong  localizer  course!),  final  monitor  controllers  with  frequency 
override  capabilities  will  issue  instructions.  Each  runway  has  a  dedicated  frequency  therefore 
you  will  not  hear  final  monitor  radio  transmissions  to  aircraft  on  the  adjacent  localizer  course. 
The  two  scenarios  requiring  final  monitor  intervention  are  aircraft  deviations  from  the  localizer 
course  centerline,  and  penetration  of  the  No  Transgression  Zone  by  the  aircraft  on  the  adjacent 
parallel  localizer  course.  For  aircraft  observed  deviating  from  the  localizer  course  centerline  or 
observed  to  overshoot  the  tum-on,  final  monitor  controllers  will  use  phraseology  such  as  "TURN 
(left/right)  AND  RETURN  TO  THE  LOCALIZER  COURSE"  or  "YOU  HAVE  CROSSED  THE 
FINAL  APPROACH  COURSE.  TURN  (left/right)  IMMEDIATELY  AND  RETURN  TO  THE 
LOCALIZER  COURSE".  When  an  aircraft  fails  to  respond  to  final  monitor  controller 
instructions  or  is  observed  penetrating  the  No  Transgression  Zone,  the  aircraft  on  the  adjacent 
parallel  localizer  course  will  be  issued  breakout  instructions  such  as  "(A/C  callsign)  TRAFFIC 
ALERT  (A/C  callsign)  TURN  (left/right)  IMMEDIATELY  HEADING  (degrees) 
CLIMB/DESCEND  AND  MAINTAIN  (altitude)".  A  descending  breakout  is  contrary  to  what 
pilots  would  normally  expect  but  the  pilot  should  be  aware  that  ATC  will  use  it  when  necessary. 
If  the  final  monitor  controller  issues  a  descending  breakout,  the  descent  will  not  take  the  pilot 
below  the  Minimum  Vectoring  Altitude  (MV A). 

WHAT  WILL  I  DO? 

To  ensure  proper  preparation  for  "what  will  I  do"  the  approach  briefing  shall  address  the 
possibility  of  an  ATC  directed  breakout.  The  briefing  should  also  include  how  that  breakout 
will  be  accomplished.  Pilots  must  comply  immediately  with  all  final  monitor  controller 
instructions.  Having  been  "cleared  for  the  approach"  pilots  are  in  a  "land  the  aircraft  mode.  For 
this  reason  it  feels  unnatural  to  be  broken  off  the  approach  particularly  if  you  have  the  localizer 
and  glideslope  "wired".  For  these  reasons,  pilots  can  be  tempted  to  question,  or  hesitate  in 
complying  with  the  final  monitor  controller’s  breakout  instructions. 

During  close  parallel  approach  operations,  immediate  execution  of  the  final  monitor  controller's 
instructions  is  mandatory.  Remember,  you  are  being  broken  off  the  approach  because  the 
aircraft  assigned  to  the  adjacent  parallel  localizer  course  has  failed  to  respond  to  the  final 
monitor  controller's  instructions  or  has  entered  the  No  Transgression  Zone—the  aircraft  is 
heading  your  way.  There  is  simply  not  enough  "real  estate"  between  you  and  the  deviating 
aircraft  to  execute  the  breakout  in  a  leisurely  manner.  For  example,  at  140  kts.  ground  speed  an 
aircraft  is  traveling  236'  per  second! 


Pilots  must  be  knowledgeable  of  aircraft  autoflight  systems  as  well  as  procedures  and  limitations. 
This  is  necessary  to  avoid  slow,  or  improper  aircraft  response  to  an  issued  clearance.  Crew 
coordination  items  must  be  thoroughly  understood  and  briefed  prior  to  commencement  of  the 
approach.  This  is  particularly  critical  for  automated  cockpits.  Remember,  during  an  ATC 
directed  breakout,  you  can  be  given  any  combination  of  turn  and/or  climb/descent  instructions. 
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PILOT  AWARENESS  OPEN  BOOK  TEST 


PILOT  AWARENESS 
OPEN  BOOK  TEST 

Pilot  Awareness  Training  for  Simultaneous  Close  Parallel  ILS  Approaches. 

“Close  Parallel”  in  describing  a  simultaneous  ILS  approach  means: 

A.  Runway  centerlines  are  less  than  4,300’  apart. 

B.  Runway  centerlines  might  be  only  3,400’  apart. 

C.  There  might  be  someone  making  an  approach  to  the  adjacent  runway  who  is  very, 
very  close. 

D.  All  of  the  above. 

If  a  pilot  is  flying  a  simultaneous  close  parallel  ILS  approach  and  the  controller  tells  him 
to  turn  off  the  localizer  the  pilot  should: 

A.  Take  his  time  because  the  passengers  don’t  like  sudden  maneuvers. 

B.  Move  the  aircraft  as  quickly  as  practical  to  avoid  a  potential  mid-air  collision. 

C.  Not  turn  off  the  localizer,  because  the  instruments  read  on  course  and  you’ve 
been  cleared  for  the  approach. 

Can  a  controller  give  a  pilot  a  descending  turn  off  the  localizer  when  the  pilot  is  on  an 
ILS  approach? 

A.  No,  not  if  the  aircraft  has  captured  the  localizer  and  glideslope. 

B.  No,  all  turns  off  the  localizer  must  be  accompanied  by  a  climb. 

C.  Yes,  provided  the  aircraft  is  not  descended  lower  than  the  minimum  vectoring 
altimde  (MV A).  If  that  is  what  it  takes  to  avoid  a  collision,  the  controller  will 
direct  a  descending  turn  off  the  localizer. 
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OPEN  BOOK  TEST 

4.  What  are  the  most  important  things  to  remember  about  simultaneous  close  parallel  ILS 

approaches? 

A.  Don’t  make  any  abrupt  turns  because  of  passenger  comfort,  and  always  question 
every  turn  off  the  ILS  localizer  given  by  ATC. 

B.  If  you  don’t  turn  immediately  off  the  localizer  when  directed  by  ATC,  perhaps 
maybe  he’ll  forget  about  you  and  you  can  get  in  on  time.  If  the  controller  is  being 
unreasonable  by  making  you  late,  stand  your  ground  and  don’t  let  him  intimidate 
you,  after  all  you’re  an  airline  captain. 

C.  There  is  probably  someone  very  close  along  side  of  you  making  an  approach  to 
the  other  runway.  If  ATC  tells  you  to  turn  off  the  localizer  it  means  that  the 
airplane  along  side  of  you  is  now  heading  your  way  and  it  might  hit  you  unless 
you  move  the  airplane  quickly. 

5.  When  you  hear  ATC  transmit  “TRAFFIC  ALERT”,  what  kind  of  message  is  going  to 

follow? 

A.  A  turn  off  the  ILS  for  someone  because  an  aircraft  on  the  parallel  runway  is 
heading  his  way. 

B.  There  is  a  new  ATIS  coming  up  and  the  controller  wants  everyone  to  listen  to  it. 

C.  The  highway  leading  into  the  airport  is  really  jammed  up  with  cars. 

6.  How  should  the  briefing  for  simultaneous  close  parallel  ILS  approaches  be  conducted? 

A.  No  briefing  is  necessary. 

B.  Use  the  standard  briefing  for  ILS  approaches. 

C.  Use  the  standard  briefing  for  ILS  approaches.  Additionally  brief  for  the  "close" 
aspect  of  the  approach,  the  possibility  of  an  ATC  directed  breakout  and  how  it 
should  be  conducted. 


[Answers  to  Questions:  1.  D  2.  B  3.  C  4.  C  5.  A  6.  C] 
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D.4  A320  BREAKOUT  PROCEDURE 

A320  BREAKOUT  PROCEDURE 
TRAINING  BULLETIN 

With  the  advent  of  closely  spaced  simultaneous  approaches,  it  has  become  apparent  that 
there  needs  to  be  a  procedure  established  for  breaking  off  from  an  ILS  before  reaching  the  D/H. 
The  present  procedures  for  "going  around"  or  "conducting  a  miss"  prior  to  the  D/H  require  the 
pilot  to  continue  to  the  missed  approach  point  before  starting  the  missed  approach  procedure 
unless  ATC  directs  otherwise.  An  ATC  directed  breakout  from  a  simultaneous  ILS  usually  is  the 
result  of  an  aircraft  from  the  adjoining  approach  course  blundering  into  the  breakout  aircraft's 
path.  The  breakout  instructions  from  ATC  can  include  a  climbing  turn,  a  level  turn,  a 
descending  turn  or  a  climb  out  straight  ahead.  Any  "breakout"  procedure  must  allow  for  these 
eventualities.  For  example,  engaging  TOGA  results  in  a  climbout  straight  ahead,  something  a 
pilot  would  not  want  to  do  if  ATC  wanted  him  to  descend  in  order  to  avoid  the  blundering 
aircraft.  The  following  procedure  was  designed  by  taking  into  consideration  all  the  breakout 
possibilities. 

1 .  Modify  existing  approach  callouts  to  include  PNF  calling  “400  feet”  as  a  crew  awareness 
advisory  when  FMA  annunciates  “LAND”. 

2.  Use  the  following  procedure  in  the  event  of  an  ATC  breakout: 

A.  Above  400  feet  AGL: 

•  Pull  heading  select  knob,  confirm  “HDG”  and  “V/S”  annimciations  on  FMA.  Set  new  ATC 
assigned  heading. 

•  Set  new  ATC  assigned  altitude  in  FCU. 

•  If  new  assigned  altitude  is  above  current  altitude,  pull  FCU  Altitude  knob  and  confirm 
“OPEN  CLIMB”  annunciation  on  FMA. 

•  If  new  assigned  altitude  is  below  current  altitude,  continue  descent  in  “V/S”  and  confirm 
“ALT  *”  and  “ALT”  annunciate  on  FMA  as  aircraft  levels  off  at  assigned  altitude. 

•  Configure  the  aircraft  appropriately. 

B.  Below  400  feet  AGL: 

•  Fly  normal  missed  approach  procedure. 

NOTE:  When  the  aircraft  is  established  on  the  ILS  and  the  LOC  and  G/S  are  annunciated,  pulling  either 
the  HEADING  or  VERTICAL  SPEED  knob  above  400  feet  radio  altimeter  causes  the  FMA  to  annunciate 
“HDG”  and  “V/S.”  The  vertical  speed  commanded  will  be  the  existing  rate  at  the  time  either  knob  is 
pulled.  Once  the  FMA  annunciates  “LAND”,  no  change  can  be  effected  with  the  autopilot  engaged  except 
“GO  AROUND.” 
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D.5  A320  BREAKOUT  PROCEDURE  OPEN  BOOK  TEST 

A320  BREAKOUT  PROCEDURE 
OPEN  BOOK  TEST 

1.  When  the  aircraft  is  established  on  the  ILS  and  LOC  and  G/S  are  annunciated,  the  pilot  not 
flying  should  call  “400  feet”  because 

a)  It  is  a  good  call  before  reaching  decision  height. 

b)  It  will  alert  the  pilot  flying  that  the  aircraft  is  the  “LAND”  mode. 

c)  This  is  the  time  to  go  “heads  up”. 

2.  When  the  aircraft  is  above  400  feet,  pulling  the  heading  select  knob  will  cause  the  FMA  to 
annunciate  which  of  the  following? 

a)  “LOC”  and  “G/S”. 

b)  Climb. 

c)  “HDG”  and  “V/S”. 

3.  The  pilot  should  use  TOGA  for  all  descending  breakouts. 

a)  False 

b)  True 

4.  When  the  aircraft  is  below  400  feet  radio  altimeter,  and  on  the  autopilot 

a)  The  heading  knob  can  be  used  to  change  heading  at  any  time. 

b)  Any  mode  can  be  changed. 

c)  Only  after  “TOGA”  is  selected,  can  the  heading  knob  be  pulled  and  a  new  heading  set. 


[Answers  to  Questions;  l.-B  2.  -  C  3. -A  4.  -C] 
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APPENDIX  E.  DATA  EXTRACTION  ALGORITHMS 


The  analysis  software  began  by  reading  an  entire  track  file  into  memory,  giving  it  access 
to  all  the  data.  The  first  task  was  to  locate  the  marker  record,  which  was  the  first  record  with  an 
event  marker  status  flag  of  “on”  and  which  signified  the  time  at  which  the  breakout  instruction 
began. 

The  next  step  was  to  crop  off  the  “bad”  data.  Bad  data  were  records  at  the  beginning 
and/or  end  of  a  track  file  that  were  not  part  of  the  flight.  They  generally  resulted  from  simulator 
actions  such  as  holding  a  fixed  position  before  a  flight  or  resetting  the  position  to  a  fixed  location 
(i.e.  on  approach  for  the  next  track  after  a  landing).  The  cropping  algorithm  looked  for  “frozen” 
positions  by  comparing  the  x  position  (distance  from  threshold)  of  various  records.  If  there  was 
a  marker  record,  the  algorithm  started  there  and  moved  towards  the  ends  of  the  tracks  until  it 
found  three  consecutive  identical  positions  (in  x).  All  records  beyond  the  three  (on  both  ends) 
were  cropped  off.  If  there  was  no  ATC  mark,  the  track  file  was  not  used  in  the  analysis. 

The  algorithms  to  measure  time-to-event  values  then  operated  on  the  cropped  track, 
identifying  numerous  events  of  interest  after  the  marker  record.  The  time  difference  between  the 
marker  record  and  the  various  event  records  gave  the  time-to-event  values.  After  the  tracks  were 
processed,  analysts  reviewed  plots  of  the  results  to  monitor  the  performance  of  the  algorithms. 
Any  unusual  results  were  investigated  and  either  explained  or  corrected. 

The  following  sections  describe  the  algorithms  used  to  identify  each  event. 

E.1  TIME  TO  ROLL 

Two  algorithms  were  used  to  identify  a  roll  starting  point.  The  algorithms  identified  the 
same  point  for  the  majority  of  tracks,  but  identified  differing  points  for  unusual  tracks. 
Nominally  the  aircraft  had  no  roll  at  the  time  of  the  breakout  instruction  and  then  several  seconds 
later  started  a  roll  to  the  left  to  make  the  turn,  as  illustrated  in  Figure  E-1. 


However,  some  tracks  were  not  nominal  and  showed  a  “double  roll”  as  illustrated  in 
Figure  E-2.  This  sometimes  occurred  when  the  pilots  started  the  roll  maneuver  before  engaging 
TOGA  thrust  during  autopilot-coupled  scenarios.  In  this  case,  the  flight  director  returned  the 
aircraft  to  the  stored  missed-approach  course.  The  pilot  then  had  to  override  the  flight  director 
and  reinitiate  the  roll  for  the  breakout  maneuver. 
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Figure  E-2.  Roll  angle  data  for  a  breakout  with  a  double  roll  Note  return  to  center  after 

TOGA  thrust  was  engaged. 


One  algorithm  found  the  first  record  in  which  the  roll  angle  was  greater  than  3  degrees  to 
the  left  (all  breakouts  were  to  the  left  in  the  study)  by  stepping  forward  from  the  marker  record 
and  checking  the  roll  angle  at  each  subsequent  record.  This  approach  was  used  on  every  track 
and  recorded  as  one  start  of  roll  determination.  For  the  example  in  Figure  F-2,  this  algorithm 
returned  a  time  to  roll  of  8.3  seconds. 

The  other  algorithm  worked  from  the  other  direction.  Starting  from  the  point  of  greatest 
roll  angle  magnitude  within  the  60  seconds  following  the  marker  record,  it  stepped  backwards 
until  the  roll  angle  crossed  3  degrees,  then  called  that  record  the  start  of  the  roll.  This  approach 
was  also  used  on  every  track,  and  the  result  was  recorded  as  the  primary  start-of-roll 
determination.  For  the  example  in  Figure  E-2,  the  algorithm  started  working  backwards  from  the 
primary  starting  point  near  27  seconds  and  returned  a  time  to  start  of  roll  of  12.3  seconds. 

When  the  start  of  roll  time  calculated  by  the  two  methods  did  not  agree,  the  data  for  that 
track  were  manually  reviewed  and  the  correct  starting  point  was  identified. 

E.2  TIMES  TO  OTHER  EVENTS 


E.2.1  Status 

Calculating  time-to-event  values  for  status-type  data  required  finding  the  first  record  after 
the  marker  record  for  which  the  value  for  the  status  had  changed.  Status-type  data  had  values  of 
either  0  or  1,  and  included  autopilot  and  TOGA  status.  The  events  of  interest  were  the 
disengagement  of  the  autopilot  and  the  engagement  of  TOGA  thrust.  There  is  an  example  of  the 
TO/GA  status  values  at  the  bottom  of  the  plot  in  Figure  E-2. 

E.2.2  Pitch,  Vertical  Speed,  and  Engine  Speed 

The  time-to-event  measurements  for  these  metrics  were  similar  to  that  of  the  first  time-to- 
roll  measurement  (stepping  forward).  There  was  no  time  limit  imposed;  each  event  could  occur 
any  time  after  the  marker  record.  For  each  metric,  a  delta  value  parameter  was  determine 
empirically  and  the  event  was  identified  when  the  data  changed  by  at  least  as  much  as  the 
parameter.  The  parameter  value  depended  on  whether  the  scenario  was  a  climbing  or  descendmg 
breakout.  Parameter  values  were  relative  to  the  value  at  the  marker  record  (start  of  ATC 
transmission).  Table  E-1  lists  the  parameters.  Although  delta-value  parameters  were  assigned 
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for  descending  breakouts,  most  pilots  did  not  change  the  vertical  speed  or  air  speed  values  in  the 
flight  director.  As  a  result,  there  were  no  pitch,  vertical  speed,  or  engine  speed  events  for  the 
majority  of  descending  breakouts. 

Table  E-1 .  Delta-Value  Parameters  for  Times  to  Event 


Metric 

Climbing 

Breakout 

Descending 

Breakout 

pitch 

+1 .2  degrees 

-1 .2  degrees 

vertical  speed 

+150  feet/minute 

-1 00  feet/minute 

engine  speed 

+300  rpm 

N/A 

Examples  of  pitch  and  vertical  speed  event  determinations  are  depicted  in  Figure  E-3. 
The  data  were  from  the  same  track  illustrated  in  Figure  E-1.  Figure  E-4  illustrates  engine  speed 
from  a  scenario  (108)  with  a  simulated  engine  failure.  Note  that  the  right  engine  speed  started 
decreasing  shortly  before  the  breakout  instruction.  This  phenomenon  was  used  to  verify  that 
there  was  a  simulated  right  engine  failure. 


5000- 

9.4000- 

1  3000- 

Q. 

0  2000- 
_C 

c 1000- 

LU 

simulated  engine  out 

Time,  sec  , 

V"  ’  left  engine 

engine  speed  increase  of  300  rpm 

t  of  ATC  instruction  (marker  record) 

- - -  right  engine 

0- 

1  ■ '  ‘  ‘  1 ' ' ' '  1 
10  -5  ( 

)  5  10  15  20  25  30  35  40 

Figure  E-4.  Engine  speed  data  during  a  breakout  with  simulated  engine  failure. 
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APPENDIX  F.  SCENARIO-ORDERED  RESULTS 


The  following  sections  present  raw  results  for  representative  performance  variables.  The 
results  are  graphically  depicted  in  the  order  of  occurrence.  The  first  (left-hand)  values  were  from 
the  first  trial  in  the  session,  and  the  last  (right-hand)  values  were  from  the  last  trial  in  the  session. 
The  lines  connect  results  for  a  given  crew.  This  aided  in  the  qualitative  assessment  for  practice 
effects  (Section  5.3.3). 

Seven  performance  variables  are  presented  for  each  phase  of  the  study:  time  to  engine 
speed  increase  (dt_engine);  time  to  pitch  increase  (dt_pitch);  time  to  increasing  vertical  speed 
(dt_vertical_speed);  time  to  min  altitude  (dt_min_altitude);  altitude  loss  for  climbing  breakouts 
only  (dz_min_altitude);  maximum  roll  angle  (maximum_roll);  and  time  to  start  of  roll  (dt_roll). 
Time  to  TOGA  thrust  input  (dt_toga)  is  also  presented  for  Phase  1.  All  times  were  measured 
from  the  start  of  the  air  traffic  controller’s  breakout  instruction  to  the  event.  Maximum  roll  angle 
was  the  largest  value  observed  during  the  breakout.  The  duration  of  the  maximum  roll  angle  was 
not  measured. 

The  data  are  coded  as  either  having  been  a  hand-flown  (HF)  approach  or  an  autopilot- 
coupled  (AP)  approach.  The  codes  along  the  horizontal  axis  represent  the  breakout  groups.  The 
code  key  is: 


Code 

Breakout  Groun 

DA 

Climbing  breakout  at  decision  altitude 

500’ 

Climbing  breakout  at  500  feet  AGL 

1000’ 

Climbing  breakout  at  1000  feet  AGL 

1800’ 

Climbing  breakout  at  1800  feet  AGL 

Descend 

Descending  breakout  at  1800  feet  AGL 

Eng.  Out 

Climbing  breakout  at  DA,  with  engine  out 
distraction 

TIME  TO  TO/GA  TIME  TO  PITCH  TIME  TO  ENGINE 

THRUST  INPUT  (s)  INCREASE  (s)  RPM  INCREASE  (s) 
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PHASE  2  SCENARIOS  IN  ORDER  OF  OCCURRENCE 
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PHASE  2  SCENARIOS  IN  ORDER  OF  OCCURRENCE 
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PHASE  2  SCENARIOS  IN  ORDER  OF  OCCURRENCE 
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APPENDIX  G.  SUMMARY  STATISTICS  FOR  ALL  TEST  VARIABLES 


Table  G-1.  DT_ROLL 

Time  from  start  of  ATC  instruction  to  start  of  roll  (seconds) 


PHASE  1 

HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

12.3 

12.6 

8.6 

3.0 

27.0 

9 

DA 

6.4 

8.7 

4.7 

3.7 

18.7, 

20 

500' 

7.2 

8.3 

4.9 

3.3 

21.0 

20 

1800' 

3.7 

4.9 

4.6 

2.7 

23.7 

19 

Descend 

CO 

9.2 

9.9 

3.1 

27.0 

9 

I  AUTOPILOT-COUPLED  APPROACHES  I 

I  Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

10.7 

16.5 

11.1 

8.7 

36.8 

6 

12.3 

14.2 

6.6 

5.3 

30.3 

20 

500' 

8.3 

10.0 

5.1 

3.3 

21.0 

19 

1800' 

5.7 

8.9 

5.8 

4.0 

23.3 

19 

Descend 

5.7 

6.7 

3.0 

4.7 

13.0 

10 

PHASE  2  _ J 

I  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

5.1 

6.6 

4.8 

3.5 

18.1 

8 

500' 

4.5 

5.1 

2.2 

2.9 

12.8 

16 

1000' 

4.8 

5.2 

1.0 

4.3 

6.7 

10 

1800' 

4.5 

5.6 

3.9 

2.7 

21.1 

19 

Descend 

5.1 

5.3 

0.9 

4.5 

6.9 

8 

A 

UTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

11.5 

12.9 

5.4 

8.8 

26.7 

10 

500' 

6.7 

7.5 

2.3 

CO 

14.4 

40 

1000' 

6.4 

6.3 

0.8 

5.1 

to 

9 

1800' 

7.2 

8.0 

2.9 

5.3 

16.8 

33 

Descend 

6.9 

7.3 

1.5 

5.6 

10.4 

9 
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Table  G-2.  MAXIMUM_ROLL 
Maximum  roll  angle  (degrees) 


1  PHASE -1  1 

\  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

17.8 

19.0 

3.0 

15.3 

25.4 

9 

25.0 

25.8 

2.6 

21.8 

32.7 

20 

500' 

25.0 

25.1 

4.7 

16.1 

36.4 

20 

1800' 

28.1 

28.7 

6.2 

18.0 

44.2 

19 

Descend 

23.9 

23.9 

3.6 

17.9 

29.0 

9 

Al 

LITOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

15.4 

14.9 

2.5 

9.4 

16.5 

7 

24.8 

25.7 

3.9 

17.0 

37.5 

20 

500' 

24.7 

25.7 

4.7 

15.3 

40.6 

19 

1800' 

24.7 

25.5 

3.7 

23.0 

40.5 

19 

Descend 

24.4 

25.6 

3.7 

23.7 

35.8 

10 

PHASE  2  _  J 

1  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

24.9 

25.1 

3.7 

18.6 

31.2 

8 

500' 

25.7 

26.8 

4.4 

20.0 

35.8 

16 

1000’ 

25.9 

27.1 

4.1 

22.2 

35.8 

10 

1800' 

28.4 

28.9 

2.9 

24.9 

35.3 

19 

Descend 

25.0 

24.5 

2.2 

21.5 

27.4 

8 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

24.9 

24.7 

0.5 

23.3 

25.1 

10 

500’ 

24.9 

25.0 

1.0 

23.8 

30.4 

40 

1000’ 

24.9 

24.9 

0.3 

24.4 

25.4 

9 

1800’ 

24.6 

24.7 

1.1 

21.2 

28.9 

33 

Descend 

24.5 

24.5 

0.1 

24.2 

24.6 

9 
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Table  G-3.  DX.TURN 

Ground  distance  traveled  from  start  of  ATC  breakout  instruction  to  start  of  roll  (feet) 


PHASE  -1 

HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

2800.3 

2844.7 

1879.4 

695.8 

5793.5 

9 

HEHI 

1537.6 

2113.8 

1149.5 

881.1 

4462.8 

20 

500' 

1729.2 

2052.7 

1232.3 

784.1 

5342.9 

20 

1800’ 

890.8 

1210.8 

1152.0 

640.6 

5868.6 

19 

Descend 

1050.8 

2348.8 

2613.0 

746.4 

7364.6 

9 

I  AUTOPILOT-COUPLED  APPROACHES 

I  Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

2436.9 

3488.2 

2434.5 

1763.3 

8407.1 

7 

2977.1 

3436.7 

1625.8 

1263.3 

7669.3 

20 

500' 

2551.0 

3095.3 

1860.3 

1059.3 

8713.7 

19 

1800' 

1508.4 

2352.6 

1533.6 

1006.5 

5770.5 

19 

Descend 

1361.3 

1611.4 

692.3 

1182.0 

3110.9 

10 

PHASE  2  _ J 

1  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

1220.9 

1602.8 

1201.3 

826.9 

4475.1  ^ 

8 

500' 

1104.4 

1220.0 

541.7 

707.3 

3170.9 

16 

1000' 

1167.7 

1256.9 

238.0 

1026.8 

1654.4 

10 

1800' 

1129.8 

1435.8 

1015.8 

725.3 

5472.1 

19 

Descend 

1284.6 

1356.3 

268.2 

1123.6 

1916.3 

8 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

2758.2 

3080.3 

1312.0 

2080.9 

6420.5 

10 

500’ 

1607.5 

1806.5 

559.7 

1027.9 

3466.4 

40 

1000' 

1518.2 

1784.2 

904.5 

1195.0 

4156.2 

9 

1800' 

1791.4 

2018.8 

738.4 

1282.2 

4198.9 

33 

Descend 

1976.3 

1897.7 

405.7 

1353.3 

2744.7 

9 

97 


Table  G-4.  DT_ENGINE 

Time  from  start  of  ATC  breakout  instruction  to  rpm  increase  (seconds) 


1  PHASE  1  _1 

HAND-FLOWN  APPROACHES 

Breakout  Group  1 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

2.8 

2.9 

0.5 

2.3 

3.7 

8 

3.8 

3.7 

0.9 

2.0 

5.3 

18 

500' 

3.7 

4.5 

3.5 

2.3 

18.0 

18 

1800' 

5.2 

5.9 

2.5 

2.7 

11.3 

18 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

1.3 

1.4 

0.3 

1.3 

2.0 

6 

2.7 

2.8 

0.8 

1.0 

4.7 

19 

500' 

4.7 

6.1 

4.1 

2.3 

17.0 

17 

1800' 

6.7 

7.8 

4.4 

1.7 

18.0 

17 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

10 

PHASE  2  1 

1  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

4.3 

5.2 

1.9  ^ 

3.7 

8.8 

8 

500' 

7.9 

9.6 

4.5 

4.3 

20.0  ^ 

16 

1000' 

11.2 

11.3 

3.2 

6.1 

17.6 

10 

1800' 

6.7 

8.1 

3.7 

cvi 

14.9 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

8 

A 

UTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

4.8 

4.7 

0.9 

3.2 

5.9 

10 

500' 

15.6 

14.3 

4.8 

3.7 

23.7 

40 

1000* 

15.2 

13.5 

3.5 

8.0 

16.8 

9 

1800' 

14.9 

12.8 

5.7 

2.1 

21.6 

31 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 
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Table  G-5.  DT_PITCH 

Time  from  start  of  ATC  breakout  instruction  to  increasing  pitch  angle  (seconds) 


PHASE  1 

HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

Enqine  Out 

3.0 

2.9 

0.7 

2.0 

4.0 

9 

DA 

3.8 

5.0 

CVJ 

2.0 

22.0 

20 

500' 

4.8 

5.1 

2.1 

2.7 

12.0 

20 

1800' 

5.3 

5.8 

2.7 

2.3 

11.9 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

3.7 

3.9 

0.7 

3.0 

5.0 

7 

4.0 

4.1 

0.8 

2.3 

6.0 

20 

500' 

5.7 

6.4 

2.4 

3.3 

12.3 

19 

1800' 

7.7 

7.6 

2.4 

4.0 

13.7 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

10 

PHASE  2  1 

1  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

4.4  ^ 

4.5 

0.7 

3.7 

6.1 

8 

500' 

6.5 

6.9 

3.0 

2.7 

14.9 

16 

1000' 

8.0 

9.1 

3.2 

6.4 

17.6 

10 

1800' 

6.4 

7.4 

3.6 

3.5 

17.3 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

8 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

6.0 

5.9 

0.8 

4.8 

6.9 

10 

500' 

10.5 

10.7 

2.7 

5.1 

18.1 

40 

1000' 

10.1 

10.1 

1.6 

6.7 

12.0 

9 

1800' 

10.7 

11.1 

3.0 

5.6 

18.5 

33 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 
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Table  G-6.  DT_VERTICAL_SPEED 

Time  from  start  of  ATC  breakout  instruction  to  start  of  climb  (seconds) 


1  PHASE -1  J 

I  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

4.0 

3.9 

0.3 

3.3 

4.3 

9 

4.7 

4.6 

1.1 

3.3 

7.0 

20 

500' 

5.2 

5.6 

2.8 

3.7 

17.0 

20 

1800' 

6.0 

7.3 

3.9 

1.7 

17.0 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

•Minimum 

Maximum 

Count 

4.0 

4.2 

0.6 

3.3 

5.0 

7 

CO 

4.5 

0.7 

2.7 

6.3 

20 

500' 

6.3 

7.1 

3.0 

4.0 

14.3 

19 

1800' 

9.0 

8.8 

3.0 

4.0 

15.0 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

10 

PHASE  2  I 

1  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

4.9 

5.0 

0.8 

4.0 

6.7 

8 

500' 

7.3 

8.2 

3.8 

3.7 

19.5 

16 

1000' 

9.2 

10.6 

3.9 

5.9 

19.2 

10 

1800' 

9.9 

9.8 

4.5 

4.5 

20.0 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

8 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

6.4 

6.3 

0.8 

5.1 

7.2 

10 

500' 

12.8 

cvi 

3.3 

5.1 

20.5 

40 

1000' 

12.3 

12.1 

2.2 

8.3 

14.9 

9 

1800' 

13.9 

14.6 

4.6 

6.9 

27.7 

33 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

Table  G-7.  DT_TOGA 

Time  from  start  of  ATC  breakout  instruction  to  TOGA  thrust  engaged  (seconds) 


PHASE  -1 

HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

4.0 

3.7 

0.9 

2.2 

5.0 

9 

DA 

3.8 

3.8 

1.0 

1.7 

5.5 

20 

500' 

3.7 

4.8 

3.7 

2.7 

19.3 

19 

1800' 

6.2 

7.3 

3.4 

3.0 

16.0 

16 

Descend 

2.7 

2.7 

N/A 

2.7 

2.7 

1 

Al 

JTOPILOT-COUPLED  APPROACHES 

I  Breakout  Group  I 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

2.7 

2.5 

0.7 

1.3 

3.3 

7 

2.8 

2.7 

1.0 

.7 

5.3 

20 

500' 

4.7 

5.3 

2.5 

2.0 

11.7 

18 

1800' 

7.3 

8.1 

CO 

3.0 

20.3 

15 

Descend 

2.0 

2.0 

N/A 

2.0 

2.0 

1 

PHASE  2  1 

I  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

3.7 

4.8 

2.1 

3.2  ^ 

9.1 

8 

500' 

5.9 

5.7 

1.0 

4.0 

6.9 

7 

1000' 

8.3 

8.3 

3.8 

5.6 

10.9 

2 

1800' 

6.5 

7.1 

2.1 

5.3 

10.9 

6 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

A 

UTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

1  Mean 

Std  Dev 

Minimum 

Mcocimum 

Count 

DA 

4.4 

CVj 

0.9 

2.7 

5.3 

10 

500' 

5.9 

5.9 

1.4 

3.5 

8.0 

8 

1000' 

8.7 

8.7 

1.3 

7.7 

9.6 

2 

1800' 

8.8 

8.5 

1.7 

6.7 

10.7 

5 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 
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Table  G-8.  DT_MIN_ALTITUDE 

Time  from  start  of  ATC  breakout  instruction  to  minimum  altitude  (seconds) 


1  PHASE  1 

HAND-FLOWN  APPROACHES 

Breakout  GrouD 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

6.3 

6.3 

0.6 

5.3 

7.0 

9 

6.3 

7.1 

3.5 

5.0 

21.0 

20 

500' 

6.9 

8.1 

4.0 

5.3 

22.3 

20 

1800' 

10.0 

13.7 

8.4 

5.7 

33.3 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  GrouoH 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

5.3 

5.4 

0.6 

4.7 

6.3 

7 

5.3 

5.3 

0.8 

3.7 

7.3 

20 

500' 

7.3 

8.2 

3.3 

4.7 

16.7 

19 

1800' 

10.7 

10.8 

3.4 

5.3 

17.7 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

10 

PHASE  2  --  - 1 

I  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

7.2 

7.6 

1.3 

5.9 

10.1 

8 

500' 

14.2 

15.0 

5.6 

7.7 

24.8 

16 

1000' 

23.0 

21.2 

6.3 

10.4 

29.1 

10 

1800' 

24.5 

23.8 

8.7 

9.9 

37.9 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

8 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

7.2 

7.1 

0.8 

5.9 

8.0 

10 

500' 

15.5 

15.0 

3.6 

6.4 

23.2 

40 

1000' 

14.7 

14.7 

2.4 

10.1 

17.6 

9 

1800' 

17.9 

20.7 

8.2 

9.6 

38.7 

33 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

► 
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Table  G-9.  DZ_MIN_ALTITUDE 

Altitude  loss  from  start  of  ATC  breakout  instruction  to  minimum  altitude  (feet) 


1  PHASE  1  1 

1  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

65.0 

65.6 

6.2 

55.0 

77.0 

9 

68.5 

71.9 

23.1 

49.0 

152.0 

20 

500' 

75.0 

83.7 

36.0 

52.0 

222.0 

20 

1800' 

125.0 

158.0 

88.6 

64.0 

387.0 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group  I 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

56.0 

56.9 

8.6 

45.0 

69.0 

7 

59.0 

60.7 

9.8 

39.0 

85.0  - 

20 

500' 

87.0 

98.8 

44.8 

53.0 

206.0 

19 

1800' 

130.0 

136.3 

48.5 

60.0 

252.0 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

10 

PHASE  2  J 

1  HAND-FLOWN  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

76.5 

80.8 

11.8 

67.0 

101.0  ^ 

8 

500' 

139.5 

144.3 

45.2 

77.0 

248.0 

16 

1000' 

195.5 

191.0 

42.1 

120.0 

253.0 

10 

1800' 

215.0 

228.5 

79.9 

104.0 

382.0 

19 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

8 

AUTOPILOT-COUPLED  APPROACHES 

Breakout  Group 

Median 

Mean 

Std  Dev 

Minimum 

Maximum 

Count 

DA 

86.0 

84.2 

10.6 

66.0 

98.0 

10 

500' 

184.0 

181.4 

49.8 

74.0 

286.0 

40 

1000' 

174.0 

180.8 

33.6 

122.0 

223.0 

9 

1800' 

230.0 

265.8 

100.7 

120.0 

477.0 

33 

Descend 

N/A 

N/A 

N/A 

N/A 

N/A 

9 
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APPENDK  H.  PHASE  1  BREAKOUT  TRAJECTORIES 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(b) 


Figure  H-l.  Phase  I  climbing  breakouts  at  decision  altitude, 
(a)  Ground  track,  (b)  Vertical  profile. 
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RELATIVE  CROSSRANGE  (ft) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


Figure  H-2.  Phase  1  climbing  breakouts  at  500 feet  above  ground  level, 
(a)  Ground  track,  (b)  Vertical  profile. 
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RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(b) 


Figure  H-3.  Phase  I  climbing  breakouts  at  1800 feet  above  ground  level 
(a)  Ground  track,  (b)  Vertical  profile. 
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RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


Figure  H-4.  Phase  1  descending  breakouts  at  1800 feet  above  ground  level 
(a)  Ground  track,  (b)  Vertical  profile. 
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RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(b) 


Figure  H-5.  Phase  1  climbing  breakouts  at  decision  altitude  with  engine  out. 
(a)Grour^  track,  (b)  Vertical  profile. 
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RELATIVE  CROSSRANGE  (ft) 


APPENDIX  I 


PHASE  2  BREAKOUT  TRAJECTORIES 


Figure  Phase  2  clintbitig  breakouts  at  decision  altitude, 
(a)  Ground  track,  (b)  Vertical  profile. 


Ill 


RELATIVE  ALTITUDE  (ft)  RELATIVE  CROSSRANGE  (ft) 


J;  'V 

*  •  ••ft 


-8000  -6000  -A 

RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 


\  ‘A 

\\  W 


-8000  -6000  -4000  -2000  0 

RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 


Figure  1-2.  Phase  2  climbing  breakouts  at  500 feet  above  ground  level 
(a)  Ground  track,  (b)  Vertical  profile. 


RELATIVE  CROSSRANGE  (ft) 


-10000  -8000  -6000  -4000  -2000  0  2000 

RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


Figure  1-3.  Phase  2  climbing  breakouts  at  1000 feet  above  ground  level 
(a)  Ground  track,  (b)  Vertical  profile. 
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RELATIVE  ALTITUDE  (ft)  RELATIVE  CROSSRANGE  (ft) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(b) 


Figure  1-4.  Phase  2  climbing  breakouts  at  1800 feet  above  ground  level 
(a)  Ground  track,  (b)  Vertical  profile. 


RELATIVE  ALTITUDE  (ft)  RELATIVE  CROSSRANGE  (ft) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(a) 


RANGE  RELATIVE  TO  POSITION  AT  START  OF  ATC  INSTRUCTION  (ft) 

(b) 


Figure  1-5.  Phase  2  descending  breakouts  at  1800 feet  above  ground  level, 
(a)  Ground  track,  (b)  Vertical  profile. 
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